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Abstract 


The  Effects  Based  Assessment  Support  System  (EBASS)  is  a  distributed  operational 
assessment  tool  based  on  the  principles  of  Value  Focused  Thinking  (VFT)  developed  jointly  by 
the  Operations  Research  Center  of  Excellence  (ORCEN)  and  the  Information  Technology  and 
Operations  Center  (ITOC)  at  the  US  Military  Academy.  Its  genesis  is  work  done  in  support  of 
the  military  command  in  Afghanistan  in  2002.  Effects  based  assessment  is  utilized  to  determine 
the  progress  of  organization  to  influence  behaviors  or  the  environment  to  achieve  a  specific  end 
state.  In  order  to  facilitate  this,  decision  makers  need  a  data  collection  and  information 
visualization  tool  flexible  enough  to  utilize  measures  most  appropriate  for  the  domain,  which  1) 
provides  a  qualitative  value  model  which  can  account  for  the  decision  makers’  most  important 
evaluation  considerations  &  measures,  and  2)  provides  quantitative  scoring  functions  and 
weights  to  evaluate  alternatives. 


About  the  Author(s) 


LTC  Ronald  C.  Dodge,  Jr.  is  a  member  of  the  United  States  Army  Acquisition 
Corps.  Currently  he  is  Assistant  Professor  at  the  United  States  Military  Academy  and 
Director  of  the  Information  Technology  and  Operations  Center  (ITOC).  Ron  received  his 
Ph.D.  in  Computer  Science  from  George  Mason  University.  His  current  research  focuses 
include  information  warfare,  network  deception,  security  protocols,  and  information 
assurance  education.  He  is  a  frequent  speaker  at  IA  conferences  and  has  published  many 
papers  on  IA  topics. 

LTC  Simon  R.  Goerger,  Ph.  D.,  is  currently  serving  as  an  Assistant  Professor  and 
the  Director  of  the  Operations  Research  Center  of  Excellence  in  the  Department  of  Systems 
Engineering  at  the  United  States  Military  Academy,  West  Point,  New  York.  He  earned  his 
Bachelor  of  Science  from  the  United  States  Military  Academy  in  1988  and  his  Masters  in 
Computer  Science  and  Doctorate  in  Modeling  and  Simulations  from  the  Naval  Postgraduate 
School,  Monterey,  CA  in  1998  and  2004,  respectively.  His  research  interests  include  combat 
models,  agent  based  modeling,  human  factors,  and  training  in  virtual  environments.  LTC 
Goerger  has  served  as  an  infantry  officer  with  the  6th  Infantry  Division  in  Alaska  &  Sinai, 
Egypt,  as  a  cavalry  officer  with  the  2d  Armored  Cavalry  Regiment  at  Fort  Polk,  LA  &  Port-a- 
Prince,  Haiti,  and  as  a  software  engineer  for  COMBAT5™,  the  US  Army’s  future  brigade  and 
below  analytical  model  for  the  21st  Century. 

LTC  Michael  J.  Kwinn,  Jr.,  Ph.  D.,  is  an  Associate  Professor  with  United  States 
Military  Academy’s  Department  of  Systems  Engineering  at  West  Point,  New  York.  His 
military  assignments  include  Fort  Bliss,  TX,  Germany,  Colorado,  and  South  Korea.  He 
earned  a  BS  from  USMA  in  1984,  a  MS  in  Systems  and  Industrial  Engineering  from  the 
University  of  Arizona  in  1994  and  a  PhD  from  the  University  of  Texas  Red  McCombs 
School  of  Business  in  2000.  His  research  interests  include  optimization  and  simulation  as 
well  as  combat  system  requirements  development  and  assessment  methodology  development. 
He  is  a  member  of  the  Board  of  Directors  for  the  Military  Operations  Research  Society. 

Major  Thomas  O.  Morel  is  an  Assistant  Professor  with  the  United  States  Military 
Academy’s  Department  of  Electrical  Engineering  and  Computer  Science  at  West  Point,  New 
York.  He  is  an  Adjutant  General  Corps  officer.  His  military  assignments  include  duty  as  an 


IV 


enlisted  Administrative  Specialist  in  S-l,  Quartermaster  Brigade  HQ,  Fort  Lee,  VA,  Second 
USACIDC  Command,  Heidelberg,  Germany,  and  as  Administration  NCO,  ROTC  Battalion, 
Southern  Illinois  University.  His  assignments  as  a  commissioned  officer  in  Heidelberg, 
Germany  include  Chief,  Personnel  Services  Branch,  379th  Personnel  Services  Company,  S-l 
and  Commander,  Headquarters  Detachment,  411th  Base  Support  Battalion.  He  also  served  as 
Battalion  Adjutant,  Harrisburg  Recruiting  Battalion,  Harrisburg,  PA.  MAJ  Morel  is  a 
graduate  of  PLDC,  BNCOC,  AG  Officer  Basic  and  Advanced  Courses,  and  CAS3.  He 
earned  his  B.A.,  Management  from  the  University  of  Maryland,  European  Division  in  1995 
and  a  M.S.,  Computer  Science  from  University  of  Illinois,  Urbana-Champaign  in  2000.  His 
interests  include  web  application  development. 

Major  Ernest  Y.  Wong  is  an  instructor  with  the  United  States  Military  Academy’s 
Department  of  Systems  Engineering  at  West  Point,  New  York.  He  received  his  B.S.  in 
Economics  from  USMA,  M.S.  in  Management  Science  and  Engineering  from  Stanford 
University,  and  M.A.  in  Education  from  Stanford  University.  As  a  Military  Intelligence 
officer  in  the  U.S.  Army,  he  has  served  in  a  variety  of  military  assignments  around  the  world. 
He  is  a  member  of  the  Alpha  Pi  Mu,  Phi  Kappa  Phi  and  the  Omega  Rho  honor  societies. 


Acknowledgements 


This  study  was  sponsored  by  Joint  Forces  Command  (JFCOM)  based  on  the  proposal 
entitled  “Effects  Based  Assessment  Support  System  (EBASS)”,  30  September  2004.  The 
project  was  complete  in  May  2006. 

The  authors  wish  to  thank  the  tarns  which  developed  the  Dynamic  Planning  and 
Assessment  Support  System  (D-PASS)  and  Afghanistan  National  Development  Assessment 
System  (ANDAS)  models  for  setting  the  stage  for  this  work.  Additional  thanks  goes  to  Dr 
Gregory  Parnell  and  COL(R)  William  Klimack  for  there  insights  into  valued  focused 
thinking. 

A  final  thanks  goes  to  the  staff  members  and  major  commanders  who  provided  their 
time  and  energies  to  help  develop  EBASS  and  proved  its  usefulness  through  use  in  real- 
world  operations  and  training  events  in  support  of  the  Global  War  on  Terrorism  (GWOT). 
Their  daily  dedication  and  steadfast  efforts  help  to  protect  our  way  of  life. 

Our  mention  of  these  contributors  does  not  imply  their  approval  of  our  results.  The 
opinions  contained  herein  are  the  opinions  of  the  authors  and  do  not  necessarily  reflect  those 
of  JFCOM,  the  United  States  Military  Academy,  the  United  States  Army,  or  the  Department 
of  Defense. 


VI 


Table  of  Contents 


Abstract . iii 

About  the  Author(s) . iv 

Acknowledgements . vi 

Table  of  Contents . vii 

List  of  Figures . ix 

List  of  Tables . ix 

Chapter  1:  Introduction . 10 

Chapter  2:  Background . 12 

2.1  Dynamic  Planning  and  Assessment  Support  System . 12 

2.2  Afghanistan  National  Development  Assessment  System . 14 

2.3  Effects  Based  Assessment  Support  System  (EBASS) . 16 

2.4  Value  Approach  to  Effects  Based  Operations  Assessment . 17 

2.5  Final  Model  Development . 21 

Chapter  3:  Effects  Assessment . 23 

3.1  T  ask-Effect  Interactions . 23 

3.2  Effects  and  Sub-Effects  Mathematics . 25 

Chapter  4:  The  Technology  and  its  Evolution . 25 

Chapter  5:  EBASS  Interface . 28 

Chapter  6:  Future  Direction  of  Research . 32 

Chapter  7:  Conclusion . 32 

Bibliography . 34 

Appendix  A:  List  of  Abbreviations . 36 

Appendix  B:  EBASS  User  Manual . 39 

Appendix  C:  EBASS  Software  Development . 85 

vii 


Appendix  D:  Defining  Appropriate  Metrics . 87 

Distribution  List . 94 

REPORT  DOCUMENTATION  PAGE  -  SF298 . 96 


viii 


List  of  Figures 


Figure  1.  Origins  of  EBASS . 17 

Figure  2.  Effects  Based  Hierarchy  Example . 18 

Figure  3.  Four  types  of  value  functions . 19 

Figure  4.  Discrete  Value  Transformation  Curve . 19 

Figure  5.  Opening  Page  of  the  Web-based  AND  AS . 21 

Figure  6.  Notional  Line  of  Operation  Assessment . 22 

Figure  7.  Effects-Based  Assessment  Support  System  Log-In  Screen . 29 

Figure  8.  Effects-Based  Assessment  Support  System  Top-Level  Overview  Screen . 30 

Figure  9.  Effects-Based  Assessment  Support  System  Effects  Assessment  Screen . 31 

Figure  10.  Effects-Based  Assessment  Support  System  Measure  with  Projection  Assessment 
Screen . 3 1 

List  of  Tables 

Table  1.  Processing  Requirements . 28 


IX 


Chapter  1:  Introduction 


Today,  we  lack  metrics  to  know  if  we  are  winning  or  losing  the  global 
war  on  terror.  Are  we  capturing,  killing  or  deterring  and  dissuading  more 
terrorists  every  day  than  the  madrassas  and  the  radical  clerics  are  recruiting, 
training  and  deploying  against  us? 

—  The  Honorable  Donald  Rumsfeld 

(“Rumsfeld’s  war-on-terror  memo”,  USA  Today,  Oct  22,  2003) 

The  above  statement  by  the  US  Secretary  of  Defense  is  as  significant  as  it  is  obvious.  It  is 
significant  because  we  as  a  nation  and  as  a  military  require  substantiated  assessments  to 
determine  future  courses  of  action  and  to  eventually  leave  behind  a  safe,  secure  and  stable 
Afghanistan  and  Iraq.  Just  as  most  businesses  rely  exclusively  on  metrics  to  assess  their 
productivity  and  chart  future  actions  to  achieve  their  goals,  we  should  require  nothing  less  from 
our  military  and  government.  In  application  however,  finding  the  right  metrics  and  leveraging 
them  to  help  make  better  decisions  tend  to  be  more  easily  said  than  done. 

The  difficulty  arises  from  the  consideration  of  multiple  factors  including  the  dispersion  of 
forces,  a  more  complete  understanding  of  enemy  intensions  and  capabilities,  the  complexities 
associated  with  military  and  political  goals  and  objectives,  and,  we  believe,  a  clear  and  robust 
methodology  to  support  such  an  assessment  to  ensure  the  organization  is  obtaining  the  desired 
effects. 

In  this  paper,  we  submit  a  methodology  based  on  sound  operations  research  theory  and  an 
application  to  improve  the  military  and  government’s  capabilities  to  link  metrics  to  the 
objectives  they  are  trying  to  achieve.  This  methodology  found  its  genesis  in  Afghanistan,  was 
developed  in  laboratory  environments  and  tested  in  Iraq  and  other  Operational  and  Strategic 
headquarters  throughout  the  world.  Though  this  application  and  methodology  is  still  maturing,  it 
is  doing  so  under  fire  and  is  becoming  a  force  in  the  assessment  community.  Our  application  and 
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methodology,  now  known  as  the  Effects  Based  Assessment  Support  System  (EBASS),  has  been 
jointly  developed  by  the  Operations  Research  Center  of  Excellence  (ORCEN)  and  the 
Information  and  Technology  Operations  Center  (ITOC),  both  at  the  United  States  Military 
Academy  at  West  Point. 

EBASS  is  a  unique  application  of  Value  Focused  Thinking  (Keeney,  1992)  and  the 
integration  with  a  web-based  application  to  facilitate  distributed  inputs  and  analysis.  EBASS  is 
also  beginning  to  have  a  significant  impact  on  the  Global  War  on  Terror  (GWOT).  In  short, 
EBASS  is: 

•  Web-based  application  designed  to  automate  and  document  the  management  of  data 
associated  with  effects,  measures  of  effectiveness,  and  associated  indicators; 

•  Specifically  geared  toward  action  officers  on  Regional  Combatant  Command 
(RCC)/Joint  Task  Force  (JTF)  staffs; 

•  Developed  by  the  United  States  Military  Academy  (USMA)  Information  Technology 
and  Operations  Center  (ITOC)  and  the  Operations  Research  Center  (ORCEN);  and 

•  Initial  development  in  Afghanistan  in  support  of  Operation  Enduring  Freedom  (OEF). 

Although  EBASS  is  a  tool  that  assists  in  the  assessment  of  effects  based  operations,  it  is 


NOT. 

•  A  magic  formula  for  assessment; 

•  A  replacement  for  effects-based  planning; 

•  A  replacement  for  assessment  process; 

•  Part  of  the  operational  network  assessment  (ONA);  or 

•  A  briefing  tool  for  senior  leaders. 

We  begin  this  paper  by  explaining  the  genesis  and  development  of  EBASS.  We  continue 
with  a  description  of  the  model  approach  and  describe  the  details  of  the  web-based  application. 
We  then  address  how  EBASS  helps  a  headquarters  directly  address  or  overcome  many  of  the 
difficulties  encountered  when  developing  assessment  methodologies.  We  conclude  with  a 
summary  of  the  applications  of  the  system  to  date  and  the  future  of  the  system. 
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Chapter  2:  Background 


2. 1  Dynamic  Planning  and  Assessment  Support  System 

In  early  June  2002,  the  Director  of  the  Staff  of  the  Combined  and  Joint  Task  Force- 180, 
stationed  in  Bagram,  Afghanistan  made  a  request  to  the  US  Military  Academy  (USMA)  for  five 
officers  to  fill  critical  positions  on  the  Coalition  Joint  Task  Force  (CJTF)  Planning  Staff,  CJ-5. 
The  Academy  responded  quickly  to  this  request  and  within  a  week  an  officer  from  five  different 
academic  departments  was  identified  and  had  begun  the  deployment  process.  The  specific 
mission  given  to  these  officers  was  not  specified  until  the  group  arrived  in  Afghanistan  in  mid 
June.  That  mission,  the  development  of  a  comprehensive  and  distributed  process  for  performing 
operational  assessments,  became  the  focus  not  only  for  these  five  officers  but  for  ten  faculty 
members  from  the  USMA  Department  of  Electrical  Engineering  and  Computer  Science  and  the 
Information  and  Technology  Operations  Center.  This  squad  of  additional  faculty  members,  both 
military  and  civilian,  made  significant  contributions  to  the  design  and  implementation  of  the 
fully  distributed  web-based  application  that  came  to  be  known  as  Dynamic  Planning  and 
Assessment  Support  System  (D-PASS). 

The  CJTF  staff,  and  in  particular,  Lieutenant  Colonel  James  Dickens,  the  Deputy 
Director  of  Plans,  CJTF- 180,  specified  a  number  of  important  requirements  for  the  system  to  the 
D-PASS  development  team.  These  were  requirements  they  felt  were  not  met  by  their  existing 
non-automated  process.  In  addition,  they  did  not  feel  that  their  requirements  were  met  by  other 
existing  software  systems,  including  the  CENTCOM-developed  Campaign  Analysis/Decision 
Support  System  (CA/DSS)  (Carlock,  2002).  It  should  be  noted  that  while  the  CA/DSS  did  not 
meet  the  needs  of  the  CJTF- 180  staff  many  of  the  underlying  ideas  in  D-PASS,  particularly  the 
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employment  of  a  hierarchical  arrangement  of  effects,  objectives,  and  tasks,  were  inspired  by  the 
CA/DSS  application. 

The  stated  requirement  for  D-PASS  was  that  it  “provide  an  automated  means  to  develop 
and  refine  effects-based  strategic,  operational,  and  tactical  assessments  to  support  the  planning 
process.”  (Dickens,  2003)  In  more  detail,  the  CJTF-180  staff  requested  that  the  D-PASS  team 
design  and  implement  an  application  that  supported  an  assessment  process  for  the  CJTF  staff  that 
was  rigorous,  consistent,  and  well-coordinated.  Significant  effort  was  expended  on  the  part  of  D- 
PASS  developers  in  Afghanistan  and  at  West  Point  to  ensure  that  the  resulting  application  met 
these  requirements. 

Two  early  design  decisions  proved  to  be  critical  to  the  success  of  the  project.  These 
decisions  included  using  an  uncomplicated  web  interface  and  incorporating  the  use  of  a 
relational  database.  Because  the  application  uses  a  relational  database,  the  many  important 
relationships  between  various  areas  of  assessment  could  be  readily  represented  and  captured. 
This  database  “back-end”  also  allowed  staff  officers  to  provide  greater  justification  for  their 
assessments  for  other  users  of  the  system,  particularly  the  Commander.  The  distributed  nature  of 
the  application  was  fully  facilitated  through  the  use  of  a  simple  web-interface.  This  allowed 
CJTF  staff  officers  who  were  responsible  for  assessment  to  directly  collaborate  with  their 
counterparts  in  other  staff  sections  on  the  CJTF  staff  as  they  performed  their  respective 
assessment.  If  should  be  noted  that  prior  to  the  development  of  D-PASS,  operation  assessments 
performed  by  CJTF  staff  officers  were  done  using  ad-hoc  methods  that  provided  only  limited 
justification  for  assessment  to  the  CJTF-180  Commander  and  higher  headquarters. 

Another  benefit  of  D-PASS  was  that,  in  addition  to  serving  as  an  analysis  tool,  its 
uncomplicated,  graphical  web  interface  allowed  it  to  serve  as  an  excellent  presentation  tool.  For 
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nearly  eighteen  months  the  CJTF  Commander  received  weekly  or  bi-weekly  briefings  in  which 
D-PASS  was  the  primary  presentation  medium.  This  meant  that  very  little  time  was  devoted  by 
staff  officers  translating  information  from  D-PASS  to  a  presentation  graphics  application  (i.e., 
Microsoft  PowerPoint). 

The  D-PASS  team  was  able  to  successfully  address  many  challenges  during  the 
development  process  including:  the  need  to  develop  and  test  an  unclassified  application  that 
processes  primarily  classified  data,  and  the  12-hour  time  difference  between  Afghanistan  and 
New  York.  One  challenge  that  proved  intractable  to  the  team  was  the  inability  of  the  team  to 
incorporate  quantitative  analysis  into  the  assessment  process.  There  are  clearly  areas  where  such 
analysis  would  contribute  to  a  better  overall  assessment  but,  unfortunately,  the  D-PASS  team 
was  unable  to  develop  consensus  among  the  users  of  the  system  concerning  the  manner  in  which 
this  analysis  should  be  done.  Consequently,  the  final  prototype  for  D-PASS,  while  considered  an 
unqualified  success  by  all  of  it  initial  users,  provided  almost  no  support  for  quantitative  analysis. 
2.2  Afghanistan  National  Development  Assessment  System 

When  a  new  group  assumed  command  of  the  forces  in  Afghanistan,  they  initially  used  D- 
PASS  as  their  primary  assessments  means.  On  an  assigned  day  each  week,  the  Assessment 
Working  Group  would  meet  to  discuss  one  or  more  of  the  Operational  Objectives  the  command 
was  training  to  achieve.  Later  that  week,  the  Assessments  Officer  would  present  their  results  to 
the  Effects  Targeting  Group,  usually  attended  by  the  Director  of  the  CJFT  Staff,  BG  Byron 
Bagby  or  by  the  Commanding  General,  LTG  John  Vines.  Each  week,  the  presiding  general 
officer  questioned  the  assessments  and  would  demand  to  know  what  metrics  the  Assessment 
Working  Group  was  using  to  come  to  their  conclusions. 
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By  mid- July  2003  it  had  become  clear  to  CJTF-180  that  D-PASS  alone  was  not  providing 
them  with  the  assessment  means  they  desired.  Mounting  frustrations  by  the  command  group,  as 
well  as  the  Assessment  Working  Group,  prompted  the  command  to  contact  the  ORCEN  at  the 
US  Military  Academy  to  assist  in  the  development  of  a  different  assessment  methodology. 

After  discussions  with  the  staff  and  BG  Bagby  and  analyzing  the  current  hierarchical, 
task-based  structure  in  D-PASS,  the  team  concluded  that  quantitatively  measuring  the 
accomplishment  of  tasks  did  not  directly  relate  to  the  achievement  of  effects,  or  objectives  as 
they  are  sometimes  called  (JFCOM,  2005.)  By  assessing  task  accomplishment,  the  D-PASS 
system  was  essentially  measuring  whether  an  organization  is  “doing  things  right.”  Effects 
assessment,  on  the  other  hand,  requires  measuring  whether  an  organization  is  “doing  the  right 
thing.”  The  new  command  wanted  to  shift  to  the  latter  -  and  link  their  assessment  to  metrics. 

To  measure  whether  CJTF-180  was  doing  the  right  things,  the  team  knew  they  could  not 
just  measure  the  top  level  effects.  They  were  too  complex  and  comprehensive  to  glean  enough 
insight  from  a  few  metrics,  no  matter  how  ingeniously  divined.  The  top-level  effects  had  to  be 
decomposed  into  sub-effects  and  continued  in  this  manner  until  they  could  be  directly  measured. 

In  the  development  of  this  effects  hierarchy,  the  team  worked  with  members  of  the  CJTF- 
180  staff  to  develop  the  important  sub-objectives  and  measures  of  effectiveness.  The 
development  was  accomplished  through  a  group  effort  between  the  analysts  and  the  staff.  It  is 
significant  to  include  the  staff  in  this  development  for  their  insights.  Their  involvement  also  aids 
in  “buy-in”  which  is  critical  to  long  term  usage  of  the  system.  Together  they  built  a  very 
comprehensive  hierarchy  of  effects,  sub-effects  and  objectives  and  MOEs  to  assess  the  overall 
operation  in  Afghanistan. 
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The  development  of  this  hierarchy  was  only  the  beginning.  The  most  difficult  and 
important  step  was  converting  this  hierarchy  to  an  assessment  methodology.  To  do  this,  the  team 
turned  to  “value  modeling”  (Keeney,  1992).  Value  modeling,  which  is  part  of  an  approach 
known  as  Value  Focused  Thinking,  is  generally  used  to  analyze  between  competing  alternatives 
and  is  a  common  approach  in  decision  analysis  (See  Parnell,  2003). 

This  value  modeling,  which  we  explain  in  greater  detail  in  this  paper  allowed  the  team  to 
develop  an  assessment  system  that  directly  linked  metrics  with  effects  the  command  wanted  to 
achieve.  This  hierarchy  and  the  overall  value  model  the  staff  and  the  team  developed  became 
known  as  the  Afghanistan  National  Development  Assessment  System  (AND AS)  and  was 
integrated  into  an  Excel-based  tool. 

Between  arrival  of  the  second  support  team  in  Afghanistan  and  final  ANDAS  model 
development,  a  new  commander  arrived,  LTG  David  Bamo  and  the  team  was  excited  to  brief  to 
him  their  work.  Unfortunately,  LTG  Bamo  did  not  like  the  model  and  did  not  trust  the  numbers 
the  model  produced.  He  had  many  just  criticisms  of  the  model  and  the  analytical  approach. 
Though  ANDAS  failed  to  gamer  further  support,  an  arguably  better  system  advanced  as  a  result- 
-the  Effects  Based  Assessment  Support  System  (EBASS). 

2.3  Effects  Based  Assessment  Support  System  (EBASS) 

The  team  left  Afghanistan  disappointed  by  not  discouraged  by  the  ANDAS  reception. 
They  knew  the  approach  was  solid  yet  unrefined.  The  most  obvious  refinement  was  to  address 
the  limitations  of  using  Excel  for  the  application.  Excel  did  not  allow  distributed  inputs  and 
viewing,  both  of  which  were  key  benefits  of  D-PASS. 

Upon  the  return  of  the  second  Afghanistan  team,  the  ORCEN  and  the  ITOC  at  West  Point 
began  to  integrate  the  analytical  approach  of  ANDAS  and  the  distributed  functionality  of  D- 
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PASS.  The  resulting  system  (Figure  1)  became  known  as  the  Effects  Based  Assessment  Support 
System  (EBASS).  Since  its  development,  EBASS  has  been  integrated  into  major  US  military 
commands  from  Hawaii  to  Iraq.  The  system  addresses  many  concerns  of  commanders  and  staffs 
have  in  conducting  assessments  and  it  is  the  focus  of  the  remainder  of  this  paper.  We  begin  with 
a  more  detailed  description  of  the  analytical  approach  and  follow  the  web-based  functionality 
afforded  by  the  system.  We  finish  with  our  directions. 


2.4  Value  Approach  to  Effects  Based  Operations  Assessment 

Value  Focused  Thinking  is  a  means  to  assess  the  “value”  of  a  variety  of  alternatives 
through  the  development  of  significant  metrics.  It  requires  the  development  of  a  hierarchy  which 
is  developed  through  interactions  with  the  decision  maker  and  other  actors  in  the  system.  The 
first  step  is  to  identify  the  primary  goal  of  the  problem  being  addressed.  The  goal  is  then 
decomposed  into  sub-functions  and  eventually  objectives  which  can  be  directly  measured.  A 
depiction  of  this  hierarchy  is  at  Figure  2,  below. 
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Figure  2.  Effects  Based  Hierarchy  Example1 

The  metric  is  the  “mapped”  to  a  value  score  through  the  application  of  a  value  curve.  In 
this  approach,  the  analyst  develops,  in  conjunction  with  a  client,  a  “value  curve”.  This  curve  is 
intended  to  be  the  functional  transformation  of  the  measure  of  effectiveness  to  a  value  between  0 
and  1 .  The  shape  of  these  curves  is  important  and  must  be  decided  on  based  on  the  relative  level 
of  importance  (or  client  value)  of  increasing  the  level  of  the  measure. 

The  four  basic  examples  of  the  shapes  of  the  curves  are  shown  in  Figure  3  below.  Note  in 
this  example,  all  types  of  curves  are  shown  on  one  chart.  The  analyst  would  develop  a  chart  for 
each  measure  and  use  the  curve  type  (and  parameters)  to  develop  the  appropriate  transformation 
function. 


1  Parnell,  Gregory  A.,  “Introduction  to  Decision  Analysis  using  Value  Focused  Thinking,”  presentation  as  part  of 
course  given  through  Innovative  Decisions,  Inc.,  2003,  slide  8. 
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Figure  3.  Four  types  of  value  functions2 

Though  continuous  functions  for  the  value  curves  are  preferred,  the  “curves”  can  also 
take  the  form  of  discrete  events.  Examples  are  the  development  of  a  constitution  for  a  country,  as 
was  done  in  Afghanistan.  In  this  instance,  the  discrete  events  could  be  No  draft  constitution, 
constitutional  convention  begins,  a  draft  constitution  is  formed,  draft  approved,  the  constitution 
is  approved  by  the  population.  In  this  example,  the  value  transformation  may  take  the  form 
shown  in  Figure  4,  below. 


Constitution  Development 


Figure  4.  Discrete  Value  Transformation  Curve3 


2  Parnell,  Gregory,  “Value  Focused  Thinking”,  Chapter  20  in  Methods  for  Conducting  Military  Operational 
Analysis:  Best  Practices  in  Use  Throughout  the  Department  of  Defense  Accepted  by  Editorial  Staff,  June  2004. 
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After  the  development  of  the  hierarchy  including  the  measures  and  establishment  of  the 
value  functions,  the  decision  maker  identifies  (or  the  staff  suggests)  weights  for  each  function, 
sub-function  and  eventually  each  measure.  The  weights  for  each  measure,  which  sum  to  one  for 
each  level,  are  then  multiplied  by  the  value  of  that  measure  which  results  in  a  “score”  at  each 
level.  In  the  application  of  this  approach  to  assessment  systems,  the  weights  represent  the 
commander’s  priorities  for  the  varying  effects  the  command  would  like  to  achieve. 

To  determine  the  value  at  each  parent  node  in  the  system,  the  weights  and  values  are 
multiplied  together  and  then  summed  as  shown  in  equation  1,  below.  Here  the  total  value  of  a 
given  node  j  is  given  as  V(X)  where  ^  is  the  weight  at  node  i  and  is  the  value  for  all  n 

metrics  which  are  “children”  of  the  parent  node  j. 

V(Xj)  =  Vi(xji)  Equation  l4 

<=i 

When  applied  to  an  assessment  system,  these  values  in  and  of  themselves  do  not  have 
meaning.  It  is  only  when  taken  over  a  period  of  time  when  the  trends  can  be  analyzed  for 
meaning.  In  this  way,  the  decision  maker  (and/or  the  staff)  can  use  the  trends  to  identify  methods 
which  are  being  successful  and  which  are  not.  They  can  also  use  this  information  to  help  identify 
the  overall  effect  of  a  course  of  action.  This  is  accomplished  by  estimating  the  potential  impact 
of  a  given  course  of  action  on  each  of  the  metrics.  The  new  score  at  each  parent  node  will 
provide  the  insight  to  determine  if  the  course  of  action  will  have  a  net  positive  effect  or  a  net 
negative  effect.  To  illustrate  this,  the  following  simple  example  is  used. 

Assume  that  a  unit  is  engaged  in  a  counter-insurgency  mission  and  there  are  only  two 
main  effects  they  are  trying  to  achieve.  One  is  to  provide  security  and  the  other  is  to  win  the 

3  Kwinn,  M.,  E.  Pohl,  R.,  Deckro,  G,  Kostas,  “Afghanistan  National  Development  Assessment  System”,  model 
developed  to  assess  progress  in  Afghanistan  for  CJTF-180  during  Operation  Enduring  Freedom,  October  2003. 

4  Kwinn,  et  al,  April  2004,  slide  8. 


20 


hearts  and  minds  of  the  populace.  To  counter  illegal  activity,  someone  suggests  imposing  a 
curfew  and  conducting  nighttime  roving  patrols.  This  will  have  the  effect  of  reducing  the 
nocturnal  illegal  activities  but  may  also  create  some  animosity  towards  the  security  forces  from 
the  general  populace.  By  determining  where  on  the  value  curves  the  metrics  for  illegal  activity 
and  acceptance  are  and  the  estimated  changes  will  be,  the  staff  can  make  a  recommendation  as  to 
the  overall  effect  of  applying  this  course  of  action.  We  now  turn  to  our  application  to  effects 
assessment. 

2. 5  Final  Model  Development 

The  team  initially  felt  that  there  was  sufficient  functionality  incorporated  in  the  original 
D-PASS  programming  to  integrate  the  weighted  sum  and  trend  analysis  of  the  new  system.  As 
development  progressed  however,  it  soon  became  clear  that  changing  the  original  code  would  be 
as  problematic  as  a  complete  construction  of  a  new  system. 

This  new  system  is  a  fully  distributed,  web-based  replication  of  the  functionality  of  the 
Excel-based  system.  It  offers  trend  analysis,  the  capability  to  provide  subjective  assessments  by 
subordinate  sections  and  commanders  at  all  levels  and  a  means  to  change  the  functions,  sub¬ 
functions  and  measures  without  reprogramming  the  entire  system.  Figure  5  is  a  screen  shot  of  the 
developmental  opening  page  of  the  system. 


Figure  5.  Opening  Page  of  the  Web-based  AND  AS 
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The  ease  of  viewing  of  the  status  of  the  trend  of  the  sub-functions  is  seen  in  Figure  6.  In 
this  screen  shot,  one  sees  a  notional  Line  of  Operation  (LOO)  and  the  trend  for  this  LOO.  Note 
the  inclusion  of  the  subjective  assessments  below  the  LOO  description  and  the  decomposition  of 
the  LOO  on  the  right  side  of  the  screen. 
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Figure  6.  Notional  Line  of  Operation  Assessment 

The  measures  under  the  higher  level  functions  and  LOOs  are  listed  on  the  right  hand  side 
of  the  screen.  The  user  can  enter  at  any  level  and  see  the  trend  of  the  function,  the  LOO  or  the 
measure  over  time.  This  will  allow  the  analyst  to  determine  which  effects  they  should 
recommend  the  command  focus  their  efforts  in  future  operations. 

The  measures  will  be  input  by  each  responsible  staff  section  on  their  own  web  page.  The 
sections  will  sign  into  the  system  and  can  view  the  status  of  all  the  functions,  but  can  only 
modify  the  value  of  the  measures  associated  with  their  sections.  Only  the  administrator,  usually 
the  Assessment  Team  Chief,  will  be  allowed  to  modify  the  weights,  the  functions  or  sub¬ 
functions,  and  the  actual  measures.  This  will  help  prevent  sections  from  modifying  the  weights  to 
change  the  trend  results. 


22 


Chapter  3:  Effects  Assessment 


Assessment  has  been  a  significant  part  of  most  military  operations.  It  is  directly 
addressed  in  most  military  doctrine.  For  example,  the  Air  Force  has  its  OODA  loop  (Observe, 
Orient,  Detect,  and  Assess)  (Baner,  1999).  This  loop  defines  the  process  Air  Force  pilots 
targeting  methodology.  The  Army  works  hard  to  improve  the  reliability  and  timeliness  of  Battle 
Damage  Assessment  (BDA)  to  ensure  the  effectiveness  of  their  operations  to  improve  future 
operations  (Koffinke  and  Brown,  1993).  Assessments  continue  to  play  a  huge  role  in  military 
operations  in  United  States’  Global  War  on  Terror  (GWOT),  especially  operations  in 
Afghanistan  (Operation  Enduring  Freedom  (OEF))  and  in  Iraq  (Operation  Iraqi  Freedom  (OIF)). 

Assessments  are  very  difficult  in  these  types  of  operations  and  are  increasingly  difficult 
the  higher  the  headquarters  performing  the  assessments.  Most  of  the  difficulties  stem  from  the 
recent  focus  on  “Effects  Based  Operations  (EBO)”  vice  more  “task  based”  operations  which 
characterized  actions  -  and  assessments  -  in  previous  conflicts  (Fayette,  2001). 

Based  on  our  experience  gained  from  discussing  effects  assessment  with  high-level 
operational  and  strategic  staffs,  senior  leaders  and  great  thinkers,  much  of  the  rationale  for  not 
directly  tying  metrics  to  effects  assessment  can  be  placed  into  three  broad  categories:  the 
multiple  interactions  between  tasks  and  effects,  inability  to  analytically  describe  the  relationship 
between  effects  and  sub-effects,  and  the  lack  of  trust  placed  in  quantitative  results. 

3.1  Task-Effect  Interactions 

In  World  War  II,  we  knew  that  to  defeat  Germany:  we  had  to  get  to  Berlin.  How  did  we 
know  we  were  achieving  success?  How  close  were  we  to  Berlin!  In  a  linear  battlefield  with  well 
defined  objectives,  the  tasks  required  for  success  -  and  assessment  -  become  much  clearer.  The 
current  military  environment  is  much  different.  In  the  operations  in  Iraq  and  Afghanistan,  the 
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coalition  forces  are  trying  to  achieve  difficult-to-define  objectives.  One  such  objective  could  be 
creating  stability  in  a  country.  These  objectives  are  often  sub-defined  into  effects  and  achieving 
these  effects  has  become  known  as  Effects  Based  Operations  (EBO).  Often  these  effects  have 
tremendous  interactions  which  lead  to  conflicts  between  achievement  of  different  effects  and 
objectives  stemming  from  military  operations  (Davis,  2001).  An  illustrative  example  helps 
explain  this  difficulty. 

Assume  that  there  are  two  effects  a  military  headquarters  would  like  to  achieve:  “Create 
a  safe  and  secure  environment”  and  “Achieve  acceptance  of  the  local  populace”.  In  order  to 
achieve  the  first  effect,  a  commander  might  direct  that  street  patrols  be  conducted  in  a  city  area. 
This  clearly  would  create  a  more  safe  and  secure  environment  in  the  area  the  patrols  cover.  The 
populace  however  would  not  appreciate  the  increased  military  presence  and  intrusion  on  their 
freedoms.  This  would  have  a  distinctively  negative  effect  on  the  second  effect  of  achieving 
acceptance  of  the  local  populace.  Should  the  commander  direct  the  patrols  or  not?  That  depends 
on  the  level  of  violence  in  the  region  and  other  factors. 

The  above  example  is  compounded  by  another  issue  -  the  unknown  relationship  between 
the  completion  of  a  task  and  the  effect  achieved.  This  is  often  a  data  problem.  For  example,  when 
we  were  asked  to  send  a  team  to  Afghanistan  to  develop  an  assessment  system,  a  General  Officer 
in  the  headquarters  there  stated,  “When  Home  Depot  wants  to  increase  sales  by  3%  in  an  area  the 
following  Saturday,  they  know  exactly  how  much  they  need  to  spend  on  advertising  in  the 
Thursday  paper.  I  want  that  ability  here  with  our  Information  Operations  (10)  efforts.  How  much 
effect  does  sending  out  1 000  flyers  have?”  That  insight  certainly  would  improve  our  ability  to 
achieve  some  objectives  in  Afghanistan.  Unfortunately,  not  much  marketing  research  data  was 
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available  on  Afghanistan  prior  to  2001.  Even  since  then,  the  country  has  been  in  such  a  state  of 
turmoil  any  data  to  this  end  would  be  too  tainted  to  provide  reasonable  insights. 

3.2  Effects  and  Sub-Effects  Mathematics 
Another  roadblock  in  the  development  of  metrics  for  assessment  is  the  lack  of 
understanding  of  relationships  between  high  level  objectives  and  the  sub-effects  identified  to 
achieve  the  objectives.  Because  of  the  lack  of  understanding  of  which  sub-effects  impact 
objectives  attainment  and  by  how  much,  staffs  and  commanders  tend  to  shy  away  from  defining 
a  number  -  or  equation  on  which  to  build  their  assessments. 

For  example,  when  trying  to  achieve  the  effect  of  “Ensure  a  stable  environment”,  it  is  not 
too  controversial  to  have  “Provide  a  law  enforcement  capability”  and  “Provide  an  education 
system”  as  at  least  two  of  the  sub-effects.  The  question  arises,  “how  much  of  each  do  you  need?” 
and  “How  much  more  of  one  brings  you  how  much  closer  to  achieving  a  safe  and  secure 
environment?” 

We  now  turn  to  the  computer  application  which  integrates  the  value  focused 
implementation  of  the  effects  assessment  and  then  integrate  both  concepts  into  the  Effects  Based 
Assessment  Support  System  (EBASS). 

Chapter  4:  The  Technology  and  its  Evolution 
The  D-PASS  application,  originally  written  using  the  Active  Server  Pages  (ASP) 
technology,  required  fundamental  changes  to  the  code  base  in  order  to  implement  the  objective 
evaluation  and  assessment  functionality  that  we  wanted  to  put  into  AND  AS.  Having  had 
numerous  programmers  work  on  the  code,  each  with  their  own  style  and  approach,  the  ASP  code 
base  had  become  almost  unmaintainable  and  needed  to  be  refactored  and  modified.  It  was 
determined  that  starting  from  scratch  and  implementing  the  use  of  data  models  and  objects  would 
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be  the  best  approach.  Additionally,  the  use  of  ASP  as  the  front  end  and  an  Access  database  as  the 
back  end  meant  that  the  system  was  very  platform  dependent.  These  two  factors  led  to  the 
decision  to  move  away  from  ASP  and  Access  and  create  EBASS  using  the  object-oriented 
language  Java,  and  the  JSP  technology  that  it  provided,  and  the  Struts  Framework.  The 
MySQL™  database  is  used  for  the  back  end.  The  use  of  Java  provides  an  object-oriented 
approach  that  greatly  enhances  code  maintainability  and  reuse,  provide  greater  modularity  and 
extensibility,  and  provide  the  platform  independence  desired. 

Web  applications  built  using  Java  and  Java  Server  Pages  (JSP)  require  the  use  of  an 
application  server  that  can  provide  support  for  these  technologies.  The  application  server  chosen 
for  EBASS  was  Tomcat™.  Tomcat™  is  not  only  used  as  a  servlet/JSP  container,  it  can  also  be 
used  as  a  stand-alone  web  server.  This  dual  functionality  allows  EBASS  to  use  Tomcat  to  serve 
its  dynamic  content  and  its  static  content  as  well.  Since  Tomcat™  has  also  been  ported  to  all 
major  operating  systems,  platform  independence  has  been  maintained.  Tomcat  is  also  closely 
coupled  with  the  very  popular  web  server  Apache.  If  Apache  is  currently  used  within  a  unit’s  IT 
infrastructure,  it  is  almost  trivial  to  add  the  Tomcat  functionality  that  allows  the  use  of  EBASS. 

MySQL™  is  a  widely-used,  open-source  relational  database  that  also  provides  platform 
independence  because  it  has  been  ported  to  all  the  major  operating  systems.  As  an  aside,  an  Open 
DataBase  Connectivity  (ODBC)  driver  for  MySQL™  is  freely  available  for  the  Windows 
platform.  This  driver  allows  a  user  to  view,  query,  and  develop  reports  from  the  EBASS  database 
using  Microsoft  Excel  or  Access;  tools  that  many  users  are  comfortable  using.  This  capability 
allows  user  to  create  custom  reports  or  views  without  needing  to  write  a  single  line  of  code. 
Although  EBASS  uses  MySQL™  as  its  back  end,  EBASS  has  been  written  in  such  a  way  that 
converting  it  to  use  another  database,  such  as  Oracle™  or  Sybase™,  is  trivial.  Virtually  any 
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database  that  provides  a  Java  Database  Connectivity  (JDBC)  driver  may  be  used  and  will  require 
minimal  changes  to  the  code. 

The  code  base  for  EBASS  is  based  on  the  Model-View-Controller  paradigm  and  is 
constructed  using  the  Struts  Framework.  The  Model  components  are  implemented  using  Java 
classes  and  represent  the  various  entity  models  within  EBASS,  i.e.  objectives,  effects,  etc.  The 
View  components  are  implemented  using  a  combination  of  JSP,  HyperText  Markup  Language 
(HTML),  Javascript,  and  Struts  tiles.  These  components  are  the  pages  presented  to  the  user  when 
the  application  is  used.  The  Controller  components  are  the  “brains”  of  the  application  and  consist 
of  various  Java  servlets  and  Struts  Actions.  The  servlets  and  actions  are  responsible  for  handling 
the  CRUD  (create,  read,  update,  and  delete)  activities  of  EBASS. 

Software  Requirements: 

The  basic  software  components  needed  to  support  EBASS: 

a.  Java  (JRE)  1 .5 

b.  Tomcat™  4. 1  or  higher 

c.  MySQL™  4.1 

The  EBASS  application  also  uses  a  third-party  graphing  package  (Davisor  Chart) 
to  construct  the  history  bar  charts.  This  package  is  proprietary  and  requires  a  license,  which  can 
be  obtained  through  Davisor’ s  web  site  at  www.davisor.com. 

Hardware  Requirements  (Table  1): 

The  EBASS  application  has  no  specific  hardware  requirements  and  does  not 
necessarily  need  its  own  dedicated  machine;  the  application  has  been  deployed  on  stand-alone 
servers,  individual  workstation  PCs,  and  laptops.  The  number  of  concurrent  users,  maintaining 
acceptable  system  performance  and  responsiveness,  and  the  number  of  services  running  on  the 
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host  machine  should  be  used  as  guidelines  in  determining  the  hardware  requirements  for  your 
organization. 


Table  1.  Processing  Requirements 


Item 

Minimum 

Recommended 

Server 

Pentium  III  processor  or  higher  (or 
equivalent  non-Intel  based  system) 

Pentium  IV  or  higher 

RAM 

512MB 

1GB  RAM  recommended  (with  at  least 
256MB  dedicated  to  the  Tomcat 

Server) 

Storage 

5-10  GB  depending  on  the  amount  of 
data  maintained  in  the  database  and 
the  frequency  of  formal  assessments. 
(300-500MBs  are  required  for  support 
components  and  -15MB  for  EBASS 
application  files) 

20-40  GB 

Chapter  5:  EBASS  Interface 

The  EBASS  interface  has  been  modified  numerous  times  in  an  attempt  to  meet  the 
desires  and  needs  of  the  client.  However,  it  has  some  basic  elements  which  provide  its 
fundamental  look  and  feel.  For  a  more  extensive  linkage  into  the  look,  feel,  and  operations  of 
EBASS,  refer  to  the  EBASS  Users  Manual  in  Appendix  B. 

Figure  7  is  an  example  screen  shot  of  the  standard  log-in  screen  for  the  tool.  Users 
provide  their  identification  (ID)  reference  and  password  to  gain  access  to  the  tool.  This  user  ID 
will  allow  the  user  to  gain  access  to  create,  update,  enter  data,  and/or  view  information  based  on 
the  authorizations  grated  the  specific  User  ID. 
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Figure  7.  Effects-Based  Assessment  Support  System  Log-In  Screen 
Figure  8  is  an  example  screen  shot  of  the  standard  top  level  overview  provided  each  user 

who  desires  to  view  the  current  level  of  assessment  of  the  overall  objective,  the  objective’s 

supporting  effects  (SEs),  the  supporting  sub  effects  (SSEs),  and  measures  of  effectiveness 

(MOEs).  Only  the  level  the  user  wish  to  drill  down  to  see  is  viewable.  The  objective  is  always 

shown  on  this  screen.  The  colored  boxes  represent  the  general  status  of  the  objective,  SE,  SSE, 

and  MOE.  Grey  means  no  data  has  been  inputted  for  one  or  more  of  the  indicators;  red  means  the 

aggregate  value  of  the  indicators,  MOEs,  SSEs,  or  SEs  are  below  the  pre-designated  threshold 

for  minimal  satisfactory  effect;  amber  is  above  the  minimal  threshold  for  a  satisfactory  effect  but 

below  the  adequate  effect  as  one  or  more  major  issues  still  exists;  yellow  is  above  the  adequate 

threshold  for  satisfactory  effect  but  below  the  goal  effect  level;  and  green  means  the  goal  effect 

for  that  level  has  been  achieved. 
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Figure  8.  Effects-Based  Assessment  Support  System  Top-Level  Overview  Screen 
Figure  9  is  an  example  screen  shot  of  the  information  provided  for  each  effect.  On  the 

right  side  of  the  screen  is  the  Objective  the  effect  is  the  child  of.  The  right  side  of  the  screen 

provides  a  listing  of  all  the  supporting  sub-effects  for  the  effect  and  their  current  color  code 

status.  The  color  codes  used  in  the  historical  bar  graph  (bottom  center)  and  the  sub-effects  are  the 

same  as  those  used  for  Figure  8.  The  screen  provides  information  on  the  date  lasted  updated,  any 

narrative  information  to  describe  the  reasons  for  the  current  status  or  recommendations  to 

address  and  issues  associated  with  the  effect  and  any  of  its  indicators.  This  allows  subordinate 

elements  to  provide  justifications  and  recommendations  for  short  falls  or  predictions  for  potential 

issues.  The  square  above  the  histogram  provides  the  general  color  code  status  based  on  the 

overall  goal  for  this  effect  and  is  the  same  as  that  seen  on  the  over-view  screen.  The  circle 

provides  the  status  based  on  the  predicted  progress  made  on  achieving  the  goal  for  this  effect. 

Both  the  goal  and  progress  predictors  are  set  prior  to  the  collection  and  input  of  data  in  order  to 

prevent  biasing  of  data. 
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Figure  9.  Effects-Based  Assessment  Support  System  Effects  Assessment  Screen 
Figure  10  is  an  example  screen  shot  of  the  goal  histogram  and  progress  chart  available  for 

each  effect,  SSE,  MOE,  and  indicator.  The  histogram  and  progress  chart  provide  additional 


visual  representation  of  the  trends  for  each  of  these  elements. 


Figure  10.  Effects-Based  Assessment  Support  System  Measure  with  Projection 

Assessment  Screen 


31 


The  example  EBASS  information  interface  screen  shots  are  just  one  aspect  of  the  tool. 
Two  other  components  are  the  objective,  SE,  SSE,  MOE,  and  indicator  development  screens  as 
well  as  the  indicator  data  input  screens.  For  a  more  information  on  these  aspects  of  the  tool,  refer 
to  the  EBASS  Users  Manual  in  Appendix  B. 

Chapter  6:  Future  Direction  of  Research 

The  system  is  ready  to  be  fully  deployed  in  its  web-based  configuration.  The  system  is 
nearly  fully  verified  in  its  functionality.  It  is  currently  deployed  in  Pacific  Command,  European 
Command  and  in  Iraq.  There  must  be  a  time  for  validation  of  the  system  prior  to  being  able  to 
rely  on  it  to  provide  sufficient  insights  to  direct  future  effects.  We  anticipate  this  taking  only  one 
or  two  months.  The  reason  is  that  the  command  has  been  compiling  most  of  the  data  required  and 
will  be  able  to  generate  data  for  past  months  allowing  good  insights  into  the  system. 

The  system,  as  it  is,  is  deemed  to  have  too  many  measures  to  be  useful  long  term.  It  may 
take  too  long  for  staff  sections  to  input  the  data  to  ensure  that  the  system  is  fully  maintained.  The 
LOOs  within  the  Area  of  Operation  in  Afghanistan  have  already  changed  and  will  continue  to 
change.  There  needs  to  be  a  means  to  allow  for  quick  changes  to  the  system.  There  will  never  be 
a  “best”  means  to  assess  the  situation  for  two  reasons.  First,  the  situation  changes  and  therefore 
the  system  must  change  to  provide  good  insights  to  direct  effects.  Second,  this  system  is 
designed  to  provide  insights  to  allow  the  commander  to  make  decisions,  therefore  as  the 
commander  changes  and  the  information  needed  to  make  those  decisions,  the  system  will  have  to 
change. 

Chapter  7:  Conclusion 

Developing  one  comprehensive  assessment  system  to  analyze  an  entire  theater  of 
operations  was  a  challenging  proposition  for  the  analysts.  In  the  end,  changing  the  command 
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structure  changed  the  complexion  of  the  problem  and  resulted  in  the  limited  acceptance  of  the 
product.  This  is  a  strong  system  based  directly  on  quantitative  analysis  to  provide  the 
commander  with  the  means  to  make  more  informed  decisions.  This  is  the  role  of  the  military 
analyst  in  supporting  a  combat  commander. 

The  military  analyst’s  role  is  not  always  easy.  As  this  study  shows,  there  are  some  who 
are  very  wary  of  quantitative  assessments  and  methods.  These  are  not  always  trusted  because 
there  is  a  belief  they  can  be  manipulated  and  that  they  can  actually  hide  reality.  The  analyst  must 
convince  the  commander  that  any  system  is  reliable  and  at  the  same  time  is  only  there  to  provide 
the  commander  with  the  means  to  make  more  informed  decisions. 

The  reality  is  that  we  live  in  a  political  time  and  one  where  the  media  has  unprecedented 
access  to  information  within  a  military  command.  Any  system  which  provides  definitive  results  - 
whether  they  accurately  reflect  the  situation  or  are  just  open  for  interpretation  -  can  possibly  be 
used  against  the  military  or  used  politically.  Commanders  must  be  sensitive  to  this  reality. 
Analysts  must  also  be  sensitive  to  this  reality  but  we  must  provide  the  commander  with  what  we 
believe  is  the  best  information  possible.  The  next  decision  lies  with  the  commander,  but  the  final 
decision  will  be  made  by  history. 
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Appendix  A:  List  of  Abbreviations 


A 

ANDAS 

Afghanistan  National  Development  Assessment  System 

ASP 

Active  Server  Pages 

B 

BDA 

Battle  Damage  Assessment 

C 

CA/DSS 

CENTCOM-developed  Campaign  Analysis  /  Decision  Support 

System 

CENTCOM 

Central  Command 

CJTF 

Coalition  Joint  Task  Force 

CJTF-180 

Combined  Joint  Task  Force  180  in  Afghanistan 

CRUD 

Create,  Read,  Update,  and  Delete 

D 

D-PASS 

Dynamic  Planning  and  Assessment  Support  System 

E 

EBASS 

Defense  Technical  Information  Center 

EBO 

Effects-Based  Operations 

G 

GWOT 

Global  War  on  Terror 

H 

HTML 

HyperText  Markup  Language 

I 

ID 

Identification 

10 

Information  Operations 

HOC 

Information  and  Technology  Operations  Center 

J 

J5 

Strategic  Plans  and  Policy,  Joint  Staff 

JDBC 

Java  Database  Connectivity 

JFCOM 

Joint  Forces  Command 

JSP 

Java  Server  Pages 

JTF 

Joint  Task  Force 

L 

LOO 

Lines  of  Operation 

LTC 

Lieutenant  Colonel 

M 

MAJ 

Major 

MNF-I 

Multi-National  Force  Iraq 

MOE 

Measures  of  Effectiveness 
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o 

OEF 

Operation  Enduring  Freedom 

ODBC 

Open  DataBase  Connectivity 

OIF 

Operation  Iraqi  Freedom 

ONA 

operational  network  assessment 

ORCEN 

Operations  Research  Center 

P 

PACOM 

Pacific  Command 

PMESII 

Political,  Military,  Economic,  Social,  Information,  and  Infrastructure 

POC 

Point  of  Contact 

R 

RCC 

Regional  Combatant  Command 

S 

SE 

Supporting  Effect 

SJFHQ 

Standing  Joint  Forces  Headquarters 

SSE 

Supporting  Sub  Effect 

U 

UML 

Unified  Modeling  Language 

USMA 

United  States  Military  Academy 

V 

YFT 

Value  Focused  Thinking 

his  table  is  sorted  a] 

phabetically 
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Appendix  B:  EBASS  User  Manual 


This  appendix  provides  the  EBASS  Users  Manual  to  assist  the  user  in  utilizing  EBASS. 
If  desired,  the  User’s  Manual  can  be  extracted  from  the  technical  report  for  use  as  a  standalone 
document. 
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Chapter  1  Introduction  to  EBASS 

The  Effects-Based  Assessment  Support  System  (EBASS)  evolved  from  a  system  called 
the  Distributed  Planning  and  Assessment  Support  System  (D-PASS).  D-PASS  was  created  to 
aid  Coalition  Joint  Task  Force  180  (CJTF-180)  in  the  Future  Operations  planning  for  the 
Operation  Enduring  Freedom  campaign  in  the  Coalition  Joint  Operations  Area,  Afghanistan.  It 
was  developed  by  a  team  of  officers  from  the  United  States  Military  Academy  at  West  Point, 
New  York,  who  augmented  the  CJTF  Plans  section  at  Bagram  Air  Base,  Afghanistan.  D-PASS 
was  a  semi-automated  tool  that  relied  on  input  from  staff  and  component  members.  EBASS 
evolved  from  D-PASS  based  on  requests  for  change  from  Joint  Forces  Command  (JFCOM).  In 
collaboration  with  JFCOM  and  the  Operations  Research  Center  (ORCEN)  at  The  United  States 
Military  Academy,  West  Point.  This  version  of  EBASS  has  been  deployed  to  Multi-national 
Force  -  Iraq  and  Standing  Joint  Forces  Headquarters  (SJFHQ)  in  the  Pacific  Command, 
European  Command,  Southern  Command,  and  Special  Operations  Command  areas  of  operation. 

EBASS  is  a  web-based  application  that  allows  action  officers  to  enter  data  via  the 
Internet.  The  data  is  input  at  the  Indicator  level  and  is  consolidated  and  normalized.  The 
normalized  value  is  then  propagated  up  the  hierarchy  to  the  parent  measure  of  effectiveness 
(MOE),  then  the  parent  effect  of  that  MOE,  and  finally  to  the  parent  objective.  The 
normalization  process  results  in  a  value  between  zero  and  one  that  reflects  the  relative 
effectiveness  of  a  particular  Objective,  Effect,  or  Measure  of  Effectiveness. 

The  system  currently  does  not  employ  any  expert  system  technology,  but  it  is  feasible 
that  it  could  be  included  in  future  versions.  Before  delving  into  the  details  of  EBASS  operations, 
it  is  important  for  you  to  understand  key  terms  and  concepts,  as  well  as  the  assessment 
methodology  employed  by  the  system.  The  following  terms  play  an  integral  role  in  EBASS: 

Objective  -  The  desired  end-state  as  a  result  of  an  operation. 

Effect  -  The  physical  and/or  behavioral  state  of  a  Political,  Military,  Economic, 
Social,  Information,  and  Infrastructure  (PMESII)  system  that  results  from  a  military  or 
non-military  action  or  set  of  actions. 

Measures  of  Effectiveness  (MOE)  -  Criteria  used  to  evaluation  how  actions 
have  affected  system  behaviors  or  capabilities.  The  MOEs  are  tied  to  Effects  and  effects 
assessment. 


Indicator  -  Events  which  occur  and  lend  a  degree  of  “proof’  for  an  MOE  that  an 
Effect  is  being  accomplished.  When  entering  Indicators  ensure  they  are  clear  and  concise,  similar 
to  a  newspaper  headline,  and  related  to  an  MOE. 

The  assessment  methodology  used  within  EBASS  is  built  upon  indicators,  which  are 
entered  into  the  system  and  then  linked  to  MOEs.  A  single  indicator  can  support  more  than  one 
MOE.  The  Indicators  support  MOEs,  which  support  Effects,  which  support  Objectives.  The 
assessment  at  each  level  is  based  upon  the  assessments  of  the  items  at  the  next  lower  echelon, 
with  the  Indicator  being  the  lowest  level.  Therefore,  the  assessment  of  an  Effect  is  derived  from 
the  assessment  of  the  MOEs  that  support  it.  At  regular  intervals,  a  formal  assessment  of  the 
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entire  hierarchy  is  made.  This  formal  assessment  creates  a  “snapshot”  of  the  database  and  results 
in  an  additional  entry  in  the  assessment  history  of  the  database. 

Individuals  within  a  headquarters  are  designated  as  “owners”  for  levels  within  the 
assessment  hierarchy  from  Objectives,  at  the  top,  down  to  Indicators  at  the  bottom.  The  owner  is 
responsible  for  ensuring  that  an  assessment  for  the  item  for  which  he/she  is  the  owner  has  been 
completed  within  the  designated  timeframe.  Additionally,  the  owner  supervises  the  assessments 
done  by  owners  of  lower  echelon  elements  which  support  his/her  items.  This  will  ensure  that  the 
assessment  is  rational,  that  the  indicators  which  support  the  assessment  have  been  input  into 
EBASS  and  linked  to  the  appropriate  MOE,  and  that  the  assessment  is  consistent,  from  MOE  on 
up,  within  the  appropriate  hierarchy. 
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Chapter  2  User  Accounts 

EBASS  has  three  types  of  accounts:  Each  account  type  has  specific  user  capabilities 
(Figure  3-7).  The  account  type  is  determined  by  the  EBASS  Administrator.  The  three  account 
types  are: 

Viewer:  The  Viewer  account  has  a  baseline  set  of  capabilities  which  allow  the  individual 
to  view  current  and  historical  unrestricted  assessment  information.  Viewers  cannot  add  or  change 
any  information  within  the  system. 

User:  The  User  account  is  the  most  common  and  allows  users  to  view  current  and 
historical  assessment  information,  to  include  restricted  portions,  and  make  changes  to  current 
assessment  information  (the  working  copy  of  assessment  staff  work  between  formal 
assessments). 

Administrator:  The  Administrator  account  provides  all  rights  and  privileges  available 
within  the  system.  The  EBASS  system  normally  has  only  one  administrator  account. 

All  proponents  within  an  organization  may  have  User  accounts  as  well  as  Viewer 
accounts. 

An  account  is  obtained  by  having  your  proponent  POC  send  an  email  to  the  EBASS 
Administrator.  Once  your  user  information  is  entered  you  will  receive  an  email  with  your 
userlD,  password ,  and  additional  instructions.  The  information  required  for  an  account  is  as 
follows: 

•  Account  Type:  User  or  Viewer 

•  Full  Name:  first,  last,  middle  initial 

•  Rank  or  Grade 

•  DSN  phone  number 

•  Commercial  phone  number 

•  Organization  name 

•  SIPR  Email  address 
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Chapter  3  Overview  of  Functionality 


EBASS  allows  users  to  perform  a  myriad  of  tasks  and  operations  upon  the  database.  This 
guide  provides  guidance  on  how  to  perform  these  specific  tasks.  It  does  not  provide  an 
explanation  of  the  “doctrinal”  process  by  which  organizations  can  use  EBASS  to  assist  them  in 
their  assessment  process.  This  chapter  will  focus  on  tasks  that  are  available  to  Viewers,  Users, 
and  Administrators.  Keep  in  mind  that  not  all  types  of  accounts  can  perform  all  of  the  functions 
presented  in  this  chapter.  Viewers  can  neither  view  restricted  information  nor  edit  any 
assessment  information.  Additional  administrator  account  functions  will  be  detailed  in  the  next 
chapter. 

Login 


Login  to  EBASS  by  entering  your  user  ID  and  password  on  the  initial  screen.  (Figure  3-1) 


Figure  3-1.  EBASS  Login  Screen 


Change  User  Password 

To  change  your  password,  click  the  “change  password”  button.  Follow  the  on-screen 
instructions  and  your  password  will  be  changed  immediately.  Once  you  submit  your  new 
password,  you  will  be  returned  to  the  main  login  screen.  You  must  now  use  your  new  password 
to  login. 

Forgot  My  Password 

If  you  forget  your  password,  click  the  “Forgot  My  Password”  button.  Pressing  the  button 
will  generate  an  email  addressed  to  the  EBASS  Administrator.  Type  a  short  message  to  the 
administrator  requesting  that  your  password  be  reset. 
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Once  you’ve  successfully  logged  in,  the  opening  screen  lists  the  Objectives  that  your 
organization  is  responsible  for  assessing.  If  there  are  no  Objective-level  items  assigned  to  your 
organization  the  list  will  be  blank.  (Figure  3-2) 


Figure  3-2.  Top-Level  View  -  Initial  Screen  after  Logging  In 

Once  the  user  has  logged  into  the  system,  the  individual  can  perform  any  function 
available  to  them  by  simply  clicking  one  of  the  drop-down  menus  at  the  top  of  the  screen.  The 
menus  appear  when  you  place  the  mouse  over  any  of  the  buttons  shown  in  Figure  3-3  below. 

Menu  Description 


Figure  3-3.  Main  Menu  -  Administrator-Only  Items  Are  Circled 
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The  General  Menu 


Figure  3-4.  General  Menu 


Top  Level  View:  This  option  will  list  the  Objective-level  elements  that  are  currently 
maintained  in  the  database.  This  is  the  same  as  the  list  you  received  when  you  logged  in. 

Recent  Comments:  This  option  will  show  all  of  the  comments  and  narratives  that  have 
been  added  since  the  last  formal  assessment.  For  example,  if  the  database  has  formal 
assessments  for  1  Jan  and  1  Feb,  and  the  current  assessment  you  are  working  on  is  dated  25  Feb, 
this  option  will  show  the  comments  added  between  1  Feb  and  25  Feb. 

Previous  Comments:  This  option  will  show  all  of  the  comments  and  narratives  that 
were  added  during  the  last  formal  assessment.  Continuing  with  the  example  given  in  the 
previous  paragraph,  this  option  would  return  all  the  comments  that  were  entered  between  1  Jan 
and  1  Feb. 

Last  Assessment:  This  option  will  display  a  consolidated  review  of  the  last  formal 
assessment.  (Figure  3-5) 


Figure  3-5.  View  Previous  Assessment 


Current  Assessment:  This  option  will  display  a  consolidated  review  of  the  working 
assessment.  (Figure  3-6) 
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Figure  3-6.  View  Current  Assessment 


User  Capabilities:  This  option  lists  the  capabilities  for  each  user-level  (Figure  3-7). 


UNCLAS 


Effects-Based  Assessment  Support  System 

(EBASS  v2.1) 


General 


Links 


EBASS  User  Level  Capabilities 


Viewer 

Viewing  Ability’  Only 

■  Formal  Assessments.  May  view  narratives  and 
recommendations. 

■  Entries  and  Metrics 

User 

Viewer  Capabilities  Plus: 

■  Access  to  Unrestricted  View 

■  Current  Working  Assessments 

■  Enter  Raw  Values  for  Metrics 

■  Enter  narrative  bullets  and  recommendations  for  Entries 
or  Metrics 

■  Edit  all  narrative  bullets  and  recommendations  that  are 
the  responsibility  of  the  User's  organization 

■  Unrestrict  an  Entry  or  Metric 

Administrator 

User  Capabilities  Plus: 

■  Add  Entries  or  Metrics 

■  Edit  AU  Entries  and  Metrics 

■  Create  and  Edit  Links  Betw  een  Entries  and/or  Metrics 

■  Edit  Weights  for  Function  Links 

■  Edit  Weights  and  Normalization  Factors  for  Metrics 

■  Save  a  Formal  Assessment 

■  Enter  and  Edit  User  Data 

■  Enter  or  Edit  Organizations 

UNCLAS 

EBASS  ©  2004,  D/EECS,  USMA 

Contact  Webmaster 


Figure  3-7.  User  Capabilities 
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The  Links  menu 


Figure  3-8.  Links  Menu 

My  Effects  and  MOEs:  This  option  will  list  all  Objectives,  Effects,  and  MOEs  that  your 
particular  organization  is  responsible  for  assessing. 

My  Indicators:  This  option  will  list  all  the  Indicators  that  your  particular  organization  is 
responsible  for  assessing. 

All  Effects  and  MOEs:  This  option  will  list  all  Objectives,  Effects,  and  MOEs  that  are 
contained  in  the  current  database,  regardless  of  which  organization  is  responsible  for  them. 

All  Indicators:  This  option  will  list  all  the  Indicators  that  are  contained  in  the  current 
database,  regardless  of  which  organization  is  responsible  for  them. 

Choose  Organization  ...:  This  option  will  allow  you  to  list  all  of  the  Effects,  MOEs, 
and  Indicators  for  a  particular  organization.  As  show  in  Figure  3-9  and  Figure  3-10. 


Figure  3-9.  Select  an  Organization 
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Figure  3-10.  All  Elements  for  a  Particular  Organization 


The  Logout  menu 

The  Logout  menu  will  close  your  active  session  with  the  system  and  log  you  out  of 
EBASS. 


EBASS  Basics 


Assessments 

There  are  two  types  of  assessment  within  EBASS:  Working  and  Formal. 

Working  Assessment:  This  type  of  assessment  is  the  action  officers’  working 
document  for  a  particular  item  and  shows  its  current  status.  Because  this  is  a  working  document, 
any  and  all  elements  of  the  item  may  be  changed  at  any  time.  An  assessment  is  considered 
working  until  the  EBASS  administrator  executes  a  formal  assessment  of  the  system.  At  that 
point,  the  working  assessment  becomes  formalized  and  a  new  working  assessment  is  created. 

Formal  Assessment:  A  formal  assessment  is  a  historical  snapshot  of  an  item  on  a 
particular  date  or  time.  No  changes  can  be  made  to  an  item  once  it  has  been  formalized. 
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Main  Screen  Orientation 

From  the  General  Menu,  choose  Top  Level  View,  then  click  on  one  of  the  items 
that  are  listed.  The  screen  that  appears  once  you’ve  selected  an  item  is  show  at  Figure  3-2Figure 
3-11  below. 


The  center  of  the  screen  shows  the  current  “working”  assessment  of  the  item;  in 
this  case  it  is  the  working  assessment  for  the  item  “EFF  2-2.”  The  graph  shows  the  history  of  the 
assessments  for  that  particular  item.  More  detail  about  the  current  assessment  is  discussed 
below. 


The  left  side  of  the  screen  shows  the  Navigation  and  Edit  submenu  and  the 
Superior  Elements  (parents,  if  you  will)  of  the  item.  In  this  case,  “OBJ  2”  is  the  parent,  or 
superior  element,  for  this  effect.  A  superior  element  may  be  selected  by  clicking  on  it. 

The  right  side  of  the  screen  shows  the  Subordinate  Elements  (children,  if  you 
will)  of  the  current  item.  In  this  case,  “EFF  2-2”  has  three  MOEs  (MOE  2-2-1,  MOE  2-2-2,  and 
MOE  2-2-3)  as  children.  Also  note  that  the  highlight  color  for  each  subordinate  element 
represents  its  current  color  status  such  as  red,  yellow,  green.  A  subordinate  element  may  be 
selected  by  clicking  on  it. 
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Figure  3-11.  Viewing  an  Assessment  in  “View”  Mode 
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Navigation  and  Edit  Menu 


Welcome  \L4J  Morel 
view's  date:  17  May  06 
change  date 


home 


back 


unhide  |  edit 


All  OBJ-EFF-MOEs 
All  Indicators 


Figure  3-12.  Navigation  and  Edit  Menu 


The  Navigation  and  Edit  menu  has  five  buttons  and  two  quick  links.  The  button  functions 
are  as  follows: 


Change  date:  This  button  allows  the  user  to  select  a  specific  date  for  which  they 
would  like  to  see  an  assessment.  When  pressed,  the  change  date  button  will  cause  a  calendar  to 
appear  in  a  pop-up  window.  The  user  selects  a  date  for  which  they  want  to  retrieve  an 
assessment  and  that  assessment  appears.  For  example,  if  the  user  wants  to  see  the  assessment 
done  on  1  April  2005,  the  user  would  click  the  button  and  select  the  appropriate  date.  The 
assessment  that  was  done  on  that  date,  or  the  last  assessment  done  before  that  date,  will  appear. 

Home:  Pressing  this  button  will  return  the  user  to  top  level  view. 

Back:  Pressing  this  button  will  take  the  user  back  to  the  previous  screen.  This 
button  can  be  pressed  repeatedly  to  go  back  further.  4' Note  that  pressing  the  back  button  returns 
the  user  to  the  previous  screen  without  refreshing  that  screen.  That  means  that  if  a  user  has 
updated  a  value,  saves  it,  then  continues  to  press  “back”  to  review  the  element  again,  the 
element ’s  updated  value  won ’t  necessarily  be  reflected.  The  user  will  need  to  press  refresh  in 
order  to  update  the  screen. 

Unhide:  Pressing  this  button  will  reveal  additional  information  about  the 
subordinate  elements  (Figure  3-13).  This  button  will  also  reveal  any  hidden  or  deleted  text 
contained  within  the  narratives  and  recommendations  for  the  assessment  being  viewed  (Figure 
4-2  and  Figure  4-3).  Once  pressed,  this  button  will  then  be  titled  “hide”  which  will  then  hide  the 
additional  information  once  again.  4 Note  that  the  Unhide  button  only  appears  if  you  are  the 
owner  of  the  element  being  presented  or  if  you  are  a  system  administrator. 
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Figure  3-13.  Viewing  “Hidden”  Subordinate  Element  Data 


Edit:  Pressing  this  button  will  allow  the  user  to  edit  the  working  assessment 
versus  just  viewing  it.  Once  pressed,  the  button  will  be  titled  “view”  which  will  then  return  the 
user  to  view  mode.  <-Note  that  the  Edit  button  only  appears  if  you  are  the  owner  of  the  element 
being  presented  or  if  you  are  a  system  administrator. 

Taking  a  closer  look  at  the  center  portion  of  the  screen  (Figure  3-14),  you  will  see  the 
administrative -type  data  for  the  item  (title,  last  updated  on  date,  owner,  etc.)  Additionally,  you 
can  see  the  history  of  the  assessments  for  this  item.  The  “working”  assessment  is  illustrated  by 
the  right-most  bar  on  the  graph  while  the  others  are  formal  assessments.  The  height  of  the  bar 
represents  the  item’s  current  assessed  value,  which  corresponds  to  the  colored  circle  and  assessed 
value  under  the  heading  “Current  Status”  on  the  left-hand  side  of  the  screen. 
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Figure  3-14.  Center  Portion  of  the  Assessment  Screen  in  Edit  Mode) 


The  Current  Status  color  indicates  the  color  associated  with  the  assessed  value.  The 
defaults  for  EBASS  color  statuses  are  Red  (0-0.33),  Yellow  (0.34  -  0.66)  and  Green  (>  0.66). 
The  assessed  value  is  a  value  between  0  and  1 .0  that  is  calculated  based  on  the  assessed  values  of 
any  subordinate  elements  and  the  weights  assigned  to  each  of  the  subordinate  elements.  As  an 
example,  note  the  weights  and  values  for  the  subordinate  elements  in  Figure  3-13  above. 

The  Current  Progress  color  indicates  the  current  assessment  value  relative  to  what  the 
expected  level  of  achievement  is  for  this  assessment  period.  For  example,  if  the  current 
assessment  status  is  red,  and  the  expected  level  of  achievement  for  this  date  was  red,  then  the 
Current  Progress  would  be  green;  indicating  that  our  current  assessed  level  is  exactly,  or  slightly 
higher,  than  what  was  expected  during  this  assessment  period.  However,  if  the  current  status 
was  expected  to  be  yellow  at  this  point  in  time,  and  the  current  assessment  status  is  red,  then  the 
progress  color  could  be  yellow,  meaning  that  the  actual  status  is  “slightly  behind”  the  expected 
level. 
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Historical  assessments  can  be  seen  by  clicking  the  “previous”  or  “next”  link  at  the  bottom 
of  the  screen.  Historical  assessments  can  also  be  seen  by  clicking  the  “change  date”  button  on 
the  Navigation  and  Edit  Menu  on  the  left-hand  side  of  the  screen.  A  small  calendar  will  appear. 
Navigate  to,  and  click  on,  the  desired  date.  The  assessment  whose  assessment  date  is  less  than  or 
equal  to  the  requested  date  will  be  displayed. 
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Chapter  4  Editing  Elements 

This  chapter  describes  how  to  edit  the  elements  of  the  assessment  structure. 

Editing  an  Objective,  Effect,  or  MOE 

An  objective,  effect,  or  MOE  can  be  edited  by  pressing  the  “edit”  button  on  the 
navigation  and  edit  menu  on  the  left-hand-side  of  the  screen.  This  results  in  the  screen  shown  in 
Figure  4-1. 
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The  editing  screen  allows  the  user  to  perform  the  following  actions: 

•  Progress  Color:  The  progress  color  can  be  changed  by  clicking  the  radio 
button  in  front  of  the  desired  color.  The  default  value  is  gray  (not  entered).  This  color  selection 
is  subjective  and  has  no  effect  on  the  assessed  value  of  the  element.  U  Note:  Unlike  the  assessed 
value,  which  is  propagated  from  an  indicator  up  through  the  hierarchy,  the  progress  color  does 
not  propagate  up.  This  means  that  the  progress  color  must  be  set  for  each  element 
independently. 


•  Progress  Remarks:  Remarks  may  be  entered  to  explain  or  provide 
justification  for  the  progress  color  chosen.  The  remarks  entered  here  are  only  shown  on  the 
consolidated  review  pages  of  the  last  or  current  assessment  menu  item  on  the  “General”  menu  at 
the  top  of  the  screen.  Progress  remarks  are  optional. 

•  Upload  Action  Plan:  This  option  only  appears  when  editing  an  effect.  It 
allows  the  user  to  attach  an  action  plan  to  an  element’s  current  assessment.  An  action  plan  is  a 
PowerPoint  slide(s)  that  provide(s)  an  explanation  as  to  why  the  progress  of  this  element  is  not 
green  and  what  actions  will  be  taken  to  get  it  to  green.  The  uploading  of  action  plans  is  optional. 

(Note  that  only  PowerPoint  slides  are  currently  supported  by  EBASS. 

•  Upload  New  Slides:  This  option  only  appears  when  editing  an  MOE.  It 
allows  the  user  to  attach  backup  slides  to  an  element’s  current  assessment  in  the  event  that 
additional  background  information  is  needed.  The  uploading  of  slides  is  optional  and  may  not  be 
used  in  your  organization.  4 Note  that  only  PowerPoint  slides  are  currently  supported  by 
EBASS.  ' 


•  Narratives  and  Recommendations:  Previously  entered  narratives  and 
recommendations  can  be  edited  here  and  new  narratives  and/or  recommendations  can  also  be 
added.  Narratives  are  used  to  provide  background  or  pertinent  information  that  pertains  to  the 
assessment  of  this  item.  Recommendations  are  used  for  making  recommendations  to  the  next 
higher  element  owner  as  to  your  recommended  course  of  action,  if  any.  Narratives  and 
recommendations  can  be  formatted  using  the  formatting  codes  shown  on  the  screen.  Enclosing 
text  in  curly  braces  {}  will  identify  the  text  as  “new”  information.  Enclosing  the  text  in  square 
brackets  [  ]  will  “hide”  the  comments.  This  means  that  the  text  will  not  appear  in  the  narrative 
shown  on  the  viewing  screen  unless  the  “unhide”  button  has  been  pressed.  Hidden  text  is 
displayed  in  gray.  Formatting  text  as  hidden  allows  the  action  officer  to  include  additional  notes 
and/or  remarks  that  he/she  doesn’t  necessarily  want  others  to  see.  Enclosing  the  text  in  angle 
brackets  <  >  will  strike-through  the  text  and  color  it  red.  Deleted  text  can  be  used  to  “remove” 
notes  or  remarks  that  are  no  longer  valid  but  perhaps  the  action  officer  wants  to  retain  them  for 
historical  purposes.  Figure  4-2  and  Figure  4-3  below  illustrate  how  a  narrative  appears  when  it  is 
restricted  and  unrestricted  with  various  formatting  codes  used  within  the  text. 
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Figure  4-2.  Restricted  View  of  Narratives 


Figure  4-3.  Unrestricted  View  of  Narratives  on  Top, 
Edit  Screen  on  the  Bottom 


Uploading  Slides 

The  process  for  uploading  slides  is  the  same  whether  it  is  to  upload  Action  Plan 
slides  for  an  Effect  or  to  upload  backup  slides  for  an  MOE.  The  following  steps  need  to  be  done 
to  complete  the  upload. 

1.  Click  the  “Upload  Action  Plan”  or  “Upload  New  Slides”  on  the  edit  screen 
(Figure  4-1  above). 

2.  Click  the  “Browse  . . .”  button  and  navigate  to  the  location  where  the  slides  to 
be  uploaded  are  located.  (Figure  4-4) 

3.  Select  the  file  to  be  uploaded  and  press  “Open.” 

4.  Press  “Submit.” 

5.  Once  the  successful  upload  page  appears,  press  “Return  to  ...”  to  return  to  the 
edit  screen.  (Figure  4-5) 
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Upload  Backup  Slides  for  EFF  2-2 

Select  the  file  yon  wish  to  turn-in  by  clicking  the  Browse  button.  Navigate  to  the  file  yon  want  to  upload,  select  it,  and  then  click  submit. 
DO  NOT  TYPE  THE  NAME  OF  THE  FILE  IN  THE  TEXT  FIELD  BELOW.  YOU  MUST  USE  THE  BROWSE  BUTTON  TO  SELECT  THE  FILE. 


After  you  click  the  submit  button,  the  file  will  be  uploaded  to  the  EBASS  server  and  will  be  accessible  by  a  link  when  you  view  the  assessment.  Additionally,  the  link  will 
appear  on  the  screen  when  the  consolidated  assessment  is  viewed. 


Note:  The  uploaded  file  is  renamed  to  match  the  short  title  and  assessment  id  for  which  it  was  uploaded.  If  yon  submit  a  file  that  has  the  same  name  as  a  file  that  yon  have 
uploaded  before,  the  older  file  will  be  overwritten  by  the  new  file.  Due  to  the  renaming  of  the  files  mentioned  above,  the  file  names  below  will  NOT  necessarily  match  the 
names  of  the  files  yon  submitted.  You  should  check  the  list  of  files  for  the  short  title  of  the  element  you  are  w  orking  on  and  confirm  the  date/time  of  the  file. 


I  Browse...  | 


|  Submit  ]  [  Reset  | 


T 


Step  1 :  Click  here  and  navigate  to  the 
file  to  be  uploaded. 


Step  2:  Click  here  to  upload  the  file. 
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Contact  Webmaster 


Figure  4-4.  Initial  Upload  Slides  Screen 
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Success 

Your  slides  w  ere  successfully  uploaded. 

User:  T 

Organization:  JTF519 

Note:  Double-check  the  list  of  files  below'  to  ensure  yonr  file  appears  and  that  the  size  of  the  file  yon  turned  in  is  not  0  bytes.  You  can  accidentally  submit  an  empty  file  if  you 
typed  the  file  name  in  the  textfield  rather  than  using  the  Brow  se  button  to  select  the  file.  As  mentioned  on  the  previous  screen,  the  filename  w  ill  contain  the  short  title  and 
assessment  id  of  the  element  you  are  currently  w  orking  on  and  not  necessarily  the  name  of  the  file  yon  uploaded. 

F3e  Size  Date 

AP  EFF  2-2_103.ppt  3489280  bytes  5/17/2006  5:04  PM 

|  Return  to  EFF  2-2  | 

J 

Click  here  to  return 
to  the  edit  screen. 

UNCLAS 
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Contact  Webmaster 


Ensure  the  file  name  has  the  short  title  of  the 
element  you  are  working  on,  the  file  size  is  not 
zero,  and  the  current  date/time  are  reflected.  If 
all  three  are  not  true,  try  the  upload  again. 


Figure  4-5.  Successful  Upload  Screen 
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Once  the  slides  are  uploaded,  the  edit  screen  will  reappear.  It  will  now  have  different 
button  options  for  the  slides.  You  can  now  upload  different  slides,  view  the  current  slides,  or 
delete  the  slides  (Figure  4-6). 


Figure  4-6.  Updated  Button  Panel  After  Slide  Upload 


Submitting  the  Changes 

Once  the  appropriate  data  has  been  entered  or  changed,  it  needs  to  be  submitted. 
There  are  two  submission  buttons:  Submit  Changes  &  Continue  Editing  will  submit  the  changes 
and  will  return  you  to  the  same  screen  you  are  on.  Submit  Changes  &  View  will  also  submit  the 
changes  but  will  return  to  you  the  “view”  screen  instead  of  the  current  editing  screen.  Figure  4-1 
illustrates  both  submission  buttons. 

Editing  an  Indicator 

An  indicator  is  viewed  and  edited  in  the  same  way  as  shown  above  for  objectives, 
effects,  and  MOEs.  However,  it  is  the  indicator  level  at  which  raw  values  are  input.  These 
inputs  can  be  of  two  types:  Natural-scale  and  Discrete-scale. 

Natural-scale  indicators  have  actual  values  as  inputs,  such  as  50,  3.7  or  0.  An 
additional  data  item,  Raw  Value,  appears  on  the  screen  when  a  natural-scale  indicator  is  viewed 
(Figure  4-7).  The  raw  value  represents  the  actual  value  entered  by  the  user  for  that  particular 
indicator.  For  example,  the  value  could  represent  the  percentage  of  combat-ready  vehicles 
within  a  unit  or  the  number  of  personnel  being  detained.  The  edit  screen  for  a  natural-scale 
indicator  provides  an  input  area  for  the  raw  value  and  also  shows  the  minimum  and  maximum 
values  that  can  be  entered  (Figure  4-8).  The  number  input  is  then  normalized  to  a  value  between 
0  and  1.0,  resulting  in  the  assessed  value.  The  normalized  value  is  derived  from  criteria  set  forth 
when  the  indicator  was  entered  into  the  system.  The  details  of  these  criteria  are  discussed  in 
Chapter  6. 
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d Rn"  Yalne :  315. 


00  1C  nr  rent  Status:  0.63 


□ 


Current  Progress  is: 


Figure  4-7.  Raw  Value  Indicator  for  a  Natural-Scale  Indicator 


Figure  4-8.  Edit  Screen  for  a  Natural-Scale  Indicator 

Discrete  scale  indicators  provide  the  user  with  a  drop-down  menu  to  enter  the 
raw  value  rather  than  entering  an  actual  number.  For  example,  in  Figure  4-9  below,  the  user  has 
the  option  of  choosing  “Low”,  “Med”,  or  “High.”  The  normalized  value  will  represent  the 
selection  the  user  made  such  as  0.0  for  “Low”  or  0.5  for  “Med”,  and  perhaps  1.0  for  “High.” 
Like  a  natural-scale  indicator,  the  normalized  value  is  derived  from  criteria  set  forth  when  the 
indicator  was  entered  into  the  system.  The  details  of  these  criteria  are  discussed  in  Chapter  6. 


INPUT  INDICATOR  DATA  HERE 


Choose  a  value  from 
the  drop-down  box 


High-  Barriers  around  entrances 


— Med-  PAP  cordon  around  entrances 

^3  Low-  Normal  ops _ _ _ |ubmit  Changes  &  View 

Edit  Indicator  | 


Current  Status: 

Current  Value:  High-  Barriers  around  entrances 


Figure  4-9.  Edit  Screen  for  a  Discrete-Scale  Indicator 


The  progress  color,  progress  remarks,  narratives,  and  recommendations  for 
indicators  are  updated  in  the  same  manner  as  discussed  for  objectives,  effects,  and  MOEs. 
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Chapter  5  Administrator’s  Menu  (Administrator  Only) 


A  user  identified  as  an  administrator  will  have  two  additional  buttons  on  the  main  menu 
(Figure  5-1).  The  two  additional  buttons  and  their  submenus  are  discussed  below. 


General 


Links 


(i 


Add /Ed  it/ Delete 


Administrator 


v  « 


Log  out 


Figure  5-1.  Administrator  Menu  Options 


Add/Edit/Delete 


Figure  5-2.  Add/Edit/Delete  Menu 


Effects  and  MOEs:  Selecting  this  option  will  allow  the  user  to  add  or  edit  an  objective, 
an  effect,  or  an  MOE.  The  details  to  add  and  edit  these  elements  are  discussed  in  Chapter  6.  An 
example  of  the  page  that  appears  when  this  item  is  selected  is  at  Figure  6-2. 


Effects-Based  Assessment  Support  System 

(EBASS  v2.1) 

General  Links  Add/Edit/Delete  Administrator  Log  out 


EBASS  ©  2004,  D/EECS,  USMA 

Contact  Webmaster 


Select  the  "owner"  of 
the  element  here. 
Only  organizations  that 
are  already  entered  into 
EBASS  will  appear  in 
the  drop-down  list. 


The  title  should  be 
descriptive  but  not 
lengthy. 


Select  the  appropriate 
classification  and 
caveat  information  from 
the  drop-down  lists. 


Click  here  to  add  the 
new  element  once  all 
the  data  has  been 
entered. 


Figure  6-2.  Effects  and  MOEs 
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Indicators:  Selecting  this  option  will  allow  the  user  to  add  or  edit  an  indicator.  The 
details  to  add  and  edit  indicators  are  discussed  in  Chapter  6.  An  example  of  the  page  that 
appears  when  this  item  is  selected  is  at  Figure  6-6. 

Admin  Page:  Selecting  this  option  will  allow  the  user  to  see  the  entire  hierarchy  for  a 
particular  objective,  effect,  or  MOE.  This  option  is  used  as  a  shortcut  to  allow  the  user  to  see  the 
entire  hierarchy  for  the  element  chosen  and  provides  links  for  various  options  that  can  be  taken 
on  the  items  displayed.  Figure  5-3  below  shows  the  initial  screen  that  allows  the  particular 
element  to  be  chosen. 


Once  an  item  is  selected  from  the  drop-down  list,  the  complete  hierarchy  for  that  item  is 
displayed.  Figure  5-4  below  shows  the  results  from  an  item  being  selected. 
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Figure  5-5  explains  the  various  links  available  to  the  user  on  the  hierarchy  page. 


Hovering  over  this  icon  will  display  the 
item's  title,  as  shown  below. 


I _ i _ _ 

I  Protect  U.S.  nationals,  facilities  and  interests  in  the  region. 

I  JqP:  2) _ _ _ 


OBJ  2  U  Edit  I  View  I  Admin  Edit  I  Hierarchy  Quick  Edit 


View  --  This  link  will 
take  the  user  to  the 
"view 

asssessment" 


Figure  5-5.  Hierarchy  Page  Link  Menu 
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The  hierarchy  quick  edit  link  is  a  quick  way  to  rename  elements  within  the 
hierarchy  of  an  element.  This  is  helpful  if  an  element’s  short  title  needs  to  be  changed  along 
with  all  of  its  subordinate  elements.  For  example,  if  an  effect’s  short  title  is  renamed  from 
EFF  1-1  to  EFF  1-2,  then  each  of  its  children  also  probably  need  to  have  their  short  titles 
renamed.  Figure  5-6  illustrates  the  hierarchy  edit  page. 


Figure  5-6.  Hierarchy  Edit  Page 
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Administrator 


Edit  Caveats 
Edit  ClassI  Fixations 
Edit  Entry  Levels 
Formal  Assessment 
Organizations 
Users 


Figure  5-7.  Administrator  Menu 

J/Note:  The  current  version  of  EBASS  does  NOT  allow  removal  of  caveats, 

classifications,  entry  levels,  organizations,  or  users. 


Edit  Caveats:  A  caveat  is  the  additional  notation  associated  with  a  classification. 
It  normal  indicates  to  whom  classified  information  can  be  released.  Many  commands  have  their 
own  caveats,  therefore,  caveats  may  be  edited  accordingly.  Caveats  already  entered  into  the 
system  can  be  updated  by  editing  the  precedence,  caveat  name,  and  abbreviation  as  necessary. 
Pressing  the  “Submit  Changes”  button  will  save  the  updates.  Adding  a  caveat  is  done  by 
entering  the  new  information  then  pressing  the  “add  new”  button.  Figure  3-lFigure  5-8  provides 
further  explanation  of  the  Update  /  Add  Caveats  screen. 


Update  /  Add  Caveats 


Figure  5-8.  Update/ Add  a  Caveat 
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Edit  Classifications:  This  option  allows  the  user  to  edit  previously  entered 
classifications  or  add  a  new  one.  The  process  to  add  or  edit  a  classification  is  much  like  the 
process  used  for  caveats  above.  However,  there  are  two  differences.  The  first  difference  is  that 
the  precedence  value  for  classifications  is  not  only  used  for  sorting,  it  is  used  to  signify  the  level 
of  classification  from  lowest  to  highest.  The  second  difference  is  that  classifications  have  a  color 
associated  with  them.  The  color  must  be  selected  from  the  drop-down  list  presented  on  the 
screen.  Figure  5-9  provides  further  information  regarding  the  updating  and  adding  of 
classifications. 


Update  /  Add  Classifications 


Precedence  indicates  the  level  of  classification 
from  lowest  to  highest  It  also  determines  the 
order  in  which  the  classifications  are  presented 
in  drop-down  lists. 


Classification  short  title.  This  is  the 
text  that  will  appear  on  the  top  and 
bottom  of  the  assessment  screen. 


Press  hereto  add  a 
classification. 


Figure  5-9.  Update/Add  a  Classification) 


Edit  Entry  Levels:  This  menu  option,  though  present  in  the  current  menu 
system,  should  be  disregarded  and  should  not  be  used. 

Formal  Assessment:  This  option  allows  the  EBASS  administrator  to  formalize 
an  assessment.  Formalizing  an  assessment  means  to  take  a  snapshot  of  all  the  data  in  EBASS 
and  create  an  additional  history  item  in  the  database.  Once  an  assessment  has  been  formalized,  it 
cannot  be  edited,  removed,  or  undone.  Additionally,  it  is  important  to  note  that  ALL  elements 
within  the  database  will  be  formalized. 

The  screen  that  appears  when  this  option  is  selected  is  shown  in  Figure 
5-10.  The  date  for  the  formal  assessment  is  the  first  entry  to  be  made.  This  entry  should  be  the 
end-date  of  the  assessment  period.  For  example,  today’s  date  is  3  April  2006.  A  formal 
assessment  is  to  be  done  but  the  desired  date  for  the  formal  assessment  is  1  April  2006.  The  first 
entry  would  then  be  completed  to  reflect  1  April  2006  by  selecting  the  appropriate  values  from 
the  drop-down  lists.  The  date  of  the  new  assessment  is  normally  the  same  day  or  one  day  later 
but  can  be  any  date  equal  to,  or  greater  than,  the  date  entered  as  the  formal  assessment  date. 
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The  formalized  assessment  may  take  a  minute  or  two  to  complete.  Once 
the  screen  indicates  the  formal  assessment  is  done,  the  assessment  will  now  appear  as  an  entry  in 
the  history  graph  presented  when  an  element  is  viewed.  The  date  used  for  the  formal  assessment 
will  be  the  title  for  the  new  history  entry.  Additionally,  the  “new”  assessment  date  is  used  as  the 
“Updated  On”  date  on  the  assessment  viewing  screen.  Figure  5-10  below  shows  the  formal 
assessment  screen  for  the  example  above  along  with  a  screen  shot  of  how  the  assessment  is 
reflected  on  the  history  graph. 


I.XCLAS 


Effects-  Based  Assessment  Support  System 

00141] 


Imuni]  l/Untniofl 


Vn  arc  aboil  to  aibnil  a  FORMAL  ASSESSMENT?  Ibn  wall  save  Ike  w  or  king  copy  as 
a  pmuiMl  record.  Oaco  tkU  k  doae,  all  avert  of  tkic  system  evil  be  able  to  view  it. 


New  assessment 

date  set  here. 


Note  the  "Updated 

on"  date  here. 


SU-ICT  THl  n.STI  I  OK  THIS  FORMAL  ASSESSMENT 
[2006  r]  Apnl  v  1  v  ^ 


SIUCT  THI  DA  IT  FOR  TIO  M  W  AS9C9UCNT  TO  START 
^  2006  v  Apm  «  I 


this  operation  CANNOT  BE  UNDONE! 
(pirate  be  paliral  ibis  may  take  a  frsr  vrcoadt) 

|  S'-ib’i'l  I-L  'fijl  Av.cvtmcnl  | 


EBASS  e  2004,  D.TECS,  t.'SMA 

(oatart  W  ebmaster 


Formal  assessment 
date  set  here. 


Note  the  new  entry  In 
the  history. 


Formal  Assessment  Screen 


MOE  2-1-Z 

PI. A  lrijci|i  iniiishiillinji/efiuipnienl  transport  activities 


Lpdaltd  On 
01  Apr  06 


CNner 

JTFS1P 


NtF  NaHKATUT  K  HAT!  HI  I  SI  .NTT  Kl  II 


No  Rl  [OUNIINiDAlITJNiN  HAVt  HI  I  N  Him 


Carreol  Slalos:  0.0"?  Carreal  Fn>-y.™sv  is:  " 


1W417B  SHI1II  IHItl.I1  ZHUAJI  IWtTMl 


View  Assessment  Screen 


Figure  5-10.  Formal  Assessment  Screen  and  Resulting  History  Entry 


68 

UNCLASSIFIED 


UNCLASSIFIED 


EBASS  User’s  Guide 


Organizations:  This  option  allows  the  editing  and  addition  of  organizations 
within  EBASS.  Organizations  already  entered  into  the  system  can  be  updated  by  editing  the 
short  name,  long  name,  and  supervisor’s  user  id  as  necessary.  Pressing  the  “Submit  Changes” 
button  will  save  the  updates.  Adding  an  organization  is  done  by  entering  the  new  information 
then  pressing  the  “Add  New”  button.  Figure  3-1  provides  further  explanation  of  the  Update/ Add 
Organizations  screen. 


Update !  Add  Organizations 


Org  ID  Short  Name  Loag  Name  Supervisor's  1'serID 

l  JTr519  ]  Joirt  Task  Force  519  rlWU7 

1  [SFJHOPAC  ~  Sanding  Joint  Forces  HO  ]  |dH347 

Edit  items  as  necessary 
then  press  here  to 
update  the  caveats). 

4  1  TEST  COM  |  Test  Commend  dW3«7 

5PACOM  IJS  Pacific  Command  .11  1  VI 7 

Short  Namr  I  nag  Naan*  Supervisor's  UiterlD 

New  Short  Name  Ne*  Long  Name  Choose  an  Kern  v  AdJNr»| 

t  t  t  t 


Short  name  of  the  organization. 

This  is  the  name  that  will  appear 
on  the  assessment  screens. 

Full  name  of  the 
organization. 

Choose  the  supervisor's  ID 
here. 

Press  here  to 
add  an 

The  supervisor  MUST 

organization. 

already  exist  as  a  user. 

Figure  5-11.  Update/Add  an  Organization 


Users:  This  option  allows  the  editing  and  addition  of  users  within  EBASS.  Users 
already  entered  into  the  system  can  be  updated  by  editing  the  information  as  necessary.  Adding 
a  user  is  done  by  selecting  “Choose  an  Item”  in  the  drop-down  list  and  pressing  the  “Get  Item” 
button.  The  new  user’s  information  can  then  be  entered.  Once  the  data  is  entered,  press  the 
“Add  User”  button.  Figure  5-12  provides  further  details  about  adding  or  updating  a  user.  Note 
that  when  a  user’s  information  is  edited,  the  password  box  appears  empty.  This  means  that  the 
password  needs  to  be  reset  each  time  the  user  is  updated.  Ensure  you  notify  the  user  of  any 
changes  made  and  inform  them  that  their  password  has  changed. 
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Select  “Choose  an  Item" 
to  add  a  new  user  or 
select  a  user  mine  from 
the  drop-down  list,  then 
press  “Get  Item." 


The  user  ID  is  the  ID  the 
user  will  use  to  login  to 
EBASS.  The  usual 
naming  convention  is 
lastname.flrstname 


ClicK  here  to  add  a  new 
user  or  update  an 
existing  user. 


Stlocl  a  I'iEr:  ChflOfift  an 'tarn  ¥ 


The  administrative  data 
here  is  self-explanatory. 
Ensure  you  select  the 
correct  account  type  and 
the  correct  organization. 


Enter  a  password. 
The  password  MUST  be 
at  least  3  characters  and 
should  include 
upper/lowercase  letters, 
numbers,  and  symbols. 

Note;  The  user's 
password  must  be  reset 
each  time  the  user's 
administrative  data  is 
updated. 


Figure  5-12.  Add/Update  a  User 
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Chapter  6  Creating  Elements  (Administrator  Only) 


This  chapter  discusses  actions  that  can  only  be  performed  by  a  system  administrator. 

The  EBASS  Hierarchy 

The  hierarchy  of  elements  within  EBASS  is  illustrated  in  Figure  6-1  below.  The 
root  of  the  hierarchy  is  an  objective  (OBJ).  An  OBJ  can  only  have  effects  (EFF)  as  children.  An 
EFF  in  turn,  can  only  have  measures  of  effectiveness  (MOE)  as  children.  Fastly,  an  MOE  will 
have  at  least  one  indicator  (IND)  as  a  child.  The  following  paragraphs  will  illustrate  the  creation 
of  these  elements  from  the  top  down. 


Short  Title  Naming  Convention 

Each  element  within  the  hierarchy  will  need  a  short  title  assigned  to  it.  As  the 
EBASS  Administrator,  you  should  define  and  strictly  adhere  to  a  naming  convention  for  the 
short  titles.  The  short  title  is  used  extensively  in  the  sorting  and  ordering  of  the  elements  when 
they  are  presented  on  the  screen.  A  naming  convention  also  aids  users  in  selecting  and  reviewing 
related  elements.  For  these  reasons,  some  time  should  be  spent  developing  your  naming 
convention. 
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The  most  often  used  convention  is  to  use  the  abbreviations  OBJ,  EFF,  MOE,  and 
IND  followed  by  a  single  space,  followed  by  a  number  and  a  dash  for  each  level  going  down  the 
hierarchy.  As  an  example,  the  following  table  illustrates  short  titles  that  may  be  used  for  related 
elements  from  OBJ  to  IND.  This  is  merely  an  example;  you  may  choose  whatever  convention 
you  desire. 


Table  6.1  Sample  Naming  Convention 


Short  Title 

Description 

OBJ  1 

Objective 

EFF  1-1 

First  EFF  under  OBJ  1 

EFF  1-2 

Second  EFF  under  OBJ  1 

MOE  1-1-1 

First  MOE  under  EFF  1-1 

MOE  1-1-2 

Second  MOE  under  EFF1-1 

IND  1-1-2- 1 

First  IND  under  MOE  1-1-2 

IND  1-1-2-2 

Second  IND  under  MOE  1-1-2 

Create  an  Objective,  Effect,  or  MOE 

The  creation  of  an  objective,  effect,  or  an  MOE  is  exactly  the  same.  The  following 
guidance  can  be  applied  no  matter  which  of  the  three  is  to  be  created.  The  only  differences 
between  the  three  are  as  follows: 

1.  Entry  Type:  A  drop-down  list  is  presented  on  the  screen  which  will  allow  the 
entry  type  to  be  chosen  (OBJ,  EFF,  or  MOE).  Ensure  the  appropriate  entry 
type  for  the  element  to  be  created  is  properly  selected. 

2.  Subordinate  Element  Links:  The  bottom  of  the  edit  screen  contains  the 
links  to  the  element’s  children.  The  drop-down  list  in  this  area  will  be 
populated  according  to  the  element  being  created.  For  example,  if  an 
objective  is  being  created,  only  effects  will  appear  in  the  drop-down  list.  If  an 
effect  is  being  created,  only  MOEs  will  appear  in  the  drop-down  list,  etc. 

Creation  of  any  element  within  EBASS  is  started  by  pressing  the  “Add/Edit/Delete” 
button  on  the  main  menu  at  the  top  of  the  screen.  Hovering  over  the  menu  will  produce  a 
submenu;  select  “Effects  and  MOEs”  from  the  submenu.  The  opening  edit  screen  appears  once 
the  selection  is  made.  Figure  6-2  below  provides  guidance  on  completing  the  edit  screen.  Be 
sure  to  pay  particular  attention  to  the  entry  type,  owner,  and  short  title. 
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Figure  6-2.  Initial  Screen  to  Create  an  Objective,  Effect,  or  MOE 


Add  Child  Links 

Once  the  data  has  been  entered  and  the  “Add  Entry”  button  has  been  pressed,  the 
following  screen  will  appear  (Figure  6-3).  Note  there  are  no  subordinate  elements  (children) 
linked  yet.  Figure  6-3  below  shows  the  drop-down  list  that  will  normally  appear  if  subordinate 
elements  have  been  previously  entered.  If  no  subordinate  elements  have  been  created  yet,  i.e.  the 
database  has  not  yet  been  populated,  the  drop-down  list  will  be  empty.  For  illustration  purposes, 
we  will  assume  that  subordinate  elements  have  already  been  added  to  the  database.  Figure  6-4 
illustrates  how  to  select,  add,  and  modify  the  weight  for  a  child  element. 
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Choose  an  Item:  OBJ  X...  Preserve  Public  Safety 
Entry  ID:  22 


Submit  Changes  |  View  Entry  |  NOTE:  View  Entry  button  does  not  sire  changes 


Entry  Type: 
Title: 


Short  Title: 
Classification: 
Description: 


Owner:  JTF519 


Preserve  Public  Safety 


This  is  the  first  objective  of  the  campaign.  The 
intent  of  this  objective  is  to  preserve  public 
saftey. 


Submit  Changes  I  View  Entry  I  NOTE:  View  Entry  button  does  not  save  Changes: 


Total  weight  of  child  links  is:  0.000  (total  should  equal  one) 

Subordinate  Effects 

Description 

None 

Weight 

N/A 

Choose  an  Item  v 

Add  Child 

EFF  2-1...  PLA  Does  Not  Establish  Lodgme 
EFF  2-2...  ZZ  Able  to  Defend  Itself  from 

EFF  3-1...  ZZ  Does  Not  Impede  U.S.  Milit 

The  items  that  appear  in  this 
list  will  be  dependent  on  the 
element  being  created. 

* 

UNCLAS 

Figure  6-3.  Initial  Screen  After  Adding  an  Element 


Figure  6-4.  Adding  a  Child  Link 


Additional  child  links  are  added  by  repeating  the  steps  above. 

It  should  be  noted  that  although  the  total  of  the  weights  cannot  exceed  1.0,  they 
can,  in  fact,  be  less  than  1.0.  If  the  weight  total  is  less  than  1.0,  the  actual  weight  used  for  each 


74 

UNCLASSIFIED 


UNCLASSIFIED 


EBASS  User’s  Guide 


link  in  calculating  the  assessed  value  will  be  each  link’s  weight,  relative  to  the  total  weight.  For 
example,  assume  there  are  two  child  links;  child  one  having  a  weight  of  0.4  and  child  two 
having  a  weight  of  0.2.  The  total  weight  of  the  two  children  is  0.6  (0.4  +  0.2).  Therefore,  when 
calculating  the  assessed  value,  child  one  will  have  a  weight  of  0.667  (0.4  /  0.6)  and  child  two  will 
have  a  weight  of  0.333  (0.2  /  0.6). 

Figure  6-5  below  is  the  initial  view  of  an  element  once  it  has  been  created. 


Effects-Based  Assessment  Support  System 

(EBASS  v2.1) 


Add/EdiUDelete  Administrator 


Log  out 


home  |  back  |  unhide  |  edit  | 

All  OBJ-EFF-MOEs 
All  Indicators 


The  Current  Status  is  gray 
because  no  indicators 
have  yet  to  be  added,  thus 
there  is  no  "value" 
associated  with  this 
assessment  yet 


OBJ  X 

Preserv  e  Public  Safety 


Updated  On 

17  May  06 


Owner 

JTF519 


No  Narratdts  ka\t  bein  entered 


Recommendations: 


NO  RECOMMENDATIONS  HAVE  BEEN  MADE 


Current  Status:  INCOMPLETE 


□ 


Current  Progress  is: 


o  ◄- 


Current  Status 
History 


Subordinate 

Here  is  the 

Elements 

"child"  effect  that 

was  added 

Effects 

F  2-1  Edit  | 
^PLA  Does  Not 
Establish 
Lodgment  On  ZZ 


The  Current  Progress  is 
gray  because  it  has  not 
been  set  yet 


There  is  no  history  yet, 
since  this  element  was 
just  created. 


EBASS  ©  2004,  D/EECS,  USMA 

Contact  Webmaster 


Figure  6-5.  View  of  an  Element  After  Creating  It 


Create  an  Indicator 

Just  like  the  creation  of  an  objective,  effect,  or  MOE,  creation  of  an  indicator  is 
started  by  pressing  the  “Add/Edit/Delete”  button  on  the  main  menu  at  the  top  of  the  screen. 
Hovering  over  the  menu  will  produce  a  submenu;  select  “Indicators”  from  the  submenu.  The 
opening  edit  screen  appears  once  the  selection  is  made  (Figure  6-6). 

Complete  the  administrative -type  data  for  the  indicator  much  like  was  done  for 
the  objectives,  effects,  and  MOEs.  Adhere  to  your  naming  convention  for  the  short  title.  We 
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now  need  to  decide  what  scale  type  the  indicator  will  have.  Unlike  other  elements  in  the  EBASS 
hierarchy,  indicators  allow  the  user  to  input  actual  metric  data.  There  are  two  ways  a  user  can 
input  the  data:  numeric  input  or  from  a  drop-down  list.  The  type  of  input  is  determined  by  the 
scale  type. 


Scale  Types.  There  are  two  types  of  scales  for  an  indicator:  Natural  and  Discrete. 

Natural  Scale 

Natural-scale  indicators  have  actual  values  as  inputs,  such  as  50,  3.7  or  0. 
For  example,  the  value  could  represent  the  percentage  of  combat-ready  vehicles  within  a  unit  or 
the  number  of  personnel  being  detained.  The  edit  screen  for  a  natural-scale  indicator  provides  an 
input  area  for  the  raw  value  and  also  shows  the  minimum  and  maximum  values  that  can  be 
entered.  (Figure  4-8)  The  number  input  is  then  normalized  to  a  value  between  0  and  1.0, 
resulting  in  the  assessed  value.  The  normalized  value  is  derived  from  the  indicator’s  graph.  The 
details  of  how  to  create  the  graph  are  discussed  next. 
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Figure  6-7  below  illustrates  the  editing  screen  for  the  graph  of  a 
natural-scale  indicator.  The  curve  on  the  graph  represents  the  function  that  will  be  applied  to  the 
raw  input  value  to  obtain  the  assessed  value.  The  y-axis  scale  values  start  at  0  and  continue 
to  1.0.  This  value  represents  the  assessed  value  that  will  be  returned  by  the  function,  given  the 
user’s  raw  value  input.  The  scale  along  the  x-axis  represents  the  raw  value  input  by  the  user. 
The  x-axis  scale  starts  with  the  minimum  value  and  continues  to  the  maximum  value  for  the 
indicator,  as  established  when  the  indicator  was  created. 


Scaling  Curve 


Y-axis  (  Oto  1.0 ) 

The  value  from  the  y-axis 
is  what  is  returned  by  the 
graph  as  the  assessed 
value. 


Select  the  function  type  here 


O  Step  Function 
©  Piece™ e  Linear 
O  Fitted  Cnrve  (Lagrange  polynomial} 


X-axis 

(indicator's  minimum  value  to  its  maximum  value)' 


pwl  0,0  100,0.25  200,0.5  300,0.75  400,1 


Get  Knots 
Reset 

Set  to  Default 
Set  Default  4  Add  Knot 
Set  Default  4  Del  Knot 


Type  the  knots  manually  here 

Tiie  coordinates  MUST  be  in  the  format  x?y  x?y  with  a  comma 
between  the  x  and  y  and  ONE  SPACE  BETWEEN  THE  PAIR,  where 
x  and  y  are  the  coordinates  desired 


Figure  6-7.  Editing  a  Natural-Scale  Graph 
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A  small  submenu  used  to  help  in  the  editing  of  the  curve  is  located  near 
the  X,  Y  coordinate  box.  The  submenu  is  explained  in  Figure  6-8  below. 


Pressing  the  "Get  Knots"  button  will 
show  the  x,y  coordinates  of  the  curve. 
If  the  curve  has  been  adjusted  using 
the  knots,  the  button  will  show  the 
updated  coordinates. 


The  "Reset"  button  is  used  to  reset  the 
x,y  coordinates  to  the  currently  saved 
values.  This  is  helpful  if  the  curve 
knots  have  been  adjusted 
inadvertently  or  incorrectly. 


The  "Set  to  Default"  button  will  reset 
the  curve  to  a  default  linear 
progression. 


The  "Set  Default  &  Add  (Del)  Knot" 
buttons  will  add  or  remove  a  knot  to  or 
from  the  curve  and  reset  it  to  a  default 
linear  progression. 


Figure  6-8.  Submenu  for  X,  Y  Coordinate  Editing 


There  are  three  types  of  curves  that  can  be  used:  Step  Function,  Piecewise 
Linear,  and  Fitted  Curve  (Lagrange  polynomial).  The  type  of  curve  used  is  up  to  you  as  the 
administrator.  A  particular  curve  may  model  your  function  better  than  another. 

A  sample  function,  modeled  by  all  three  curve  types,  is  shown  in  Figure  6-9. 
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Figure  6-9.  Sample  Function  Plotted  on  all  Three  Types  of  Curves 

The  assessed  value  returned  by  the  graph  is  the  value  along  the  y-axis 
where  the  raw  value  along  the  x-axis  intersects  with  the  curve  on  the  graph.  Figure  6-10 
illustrates  this  correlation. 
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1.0 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0.0 


Scaling  Curve 


Df  the  user  inputs  a  raw  value  of  ICC, 

If  the  user  inputs  a  raw  value  of  35C,  the 

the  assessed  value  returned  will  be 

assessed  value  returned  will  be 

approximately  0.25 

approximately  O.frG 

Figure  6-10.  Assessed  Value  Curve  Example 


The  graph  for  a  natural-scale  indicator  can  be  adjusted  in  two  ways.  The  first  way 
is  to  click-and-hold  one  of  the  “knots”  (small  black  squares)  on  the  curve.  Move  the  knot  to  the 
desired  position  and  release.  The  other  knots  can  be  adjusted  in  the  same  manner  until  the  curve 
matches  the  desired  pattern.  The  curve  can  easily  be  adjusted  this  way;  however,  moving  the 
knots  to  a  very  specific  location  can  be  difficult.  Manually  inputting  the  x,  y  coordinates,  the 
second  way  of  adjusting  the  knots,  is  much  more  accurate. 

The  second  way  to  adjust  the  curve  is  to  manually  edit  the  x,  y  coordinates.  There 
are  two  boxes  on  the  screen  that  show  the  coordinates  (Figure  6-11).  The  top  box  shows  the 
current  coordinates.  The  bottom  box  initially  shows  the  current  coordinates,  but  the  coordinates 
can  be  edited  in  this  box.  Read  the  instructions  on  the  screen  just  above  the  box.  It  is  VERY 
important  that  the  coordinates  be  typed  exactly  as  described.  Once  the  coordinates  have  been 
edited,  check  the  “Check  here  to  set  values  manually”  box  then  press  the  “Submit  Manual 
Knots”  button  to  submit  your  changes.  Figure  6-11  below  illustrates  the  manual  editing  of  the 
coordinates.  Note :  the  current  version  of  EBASS  does  NOT  check  for  errors  within  the 
coordinates.  If  an  error  screen  appears  when  the  coordinates  are  submitted,  press  the  back 
button  on  the  browser  and  try  again.) 
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1.0 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0.0 


pwl  0,0  100,0.25  200,0.5  300,0.75  400,1 

Get  Knots 

Reset 

Set  to  Default 

Set  Default  &  Add  Knot  | 
Set  Default  4  Del  Knot  | 


Type  the  knots  manually  here 

The  coordinates  MUST  be  in  the  format  x,v  x.y  with  a  comma 
between  the  x  and  y  and  ONE  SPACE  BETWEEN  THE  PAIR,  where 
x  and  y  are  the  coordinates  desired 

I  pwl  0,0  100,0.25  Il50, 0.35  200,0.651300,0.75  400,1  Check  here  to  set 

-  I  values  manuallv 

0 

Submit  Manual  Knots  | 

J 

The  x,y  coordinates  for  the  curve  can  be  edited  here  manually. 

Edit  the  coordinates  as  necessary,  check  the  box,  and  press  the  "Submit  Manual  Knots"  button  to 
make  the  changes. 

In  the  red  box  above,  the  coordinate  150,0.35  has  been  added  and  the  200,0.5  coordinate  has  been 
adjusted  to  200,0.65.  The  results  of  the  changes  are  shown  in  the  graph  below. 

The  format  for  the  coordinates  is:  function<space>x,y<space>x,y  Example:  pwl  0,.2  100,0.25 

functions: 

stp  =  Step  Function 

pwl  =  Piecewise  Linear 

Igr  =  Fitted  Curve  (Lagrange  polynomial) 


1.0 
0.9 
0.8 
0.7 
0.6 
0.5 
0.4 
0.3 
0.2 
0.1 
0.0 

0  50  100  150  200  250  300  350  400 


Results  from  the  coordinate  changes  above 


Current  coordinates 
(cannot  be  edited) 


Scaling  Curve 


0  50  100  150  200  250  300  350  400 

O  Step  Function 

®  Piecewise  Linear 

O  Fitted  Curve  (Lagrange  polynomial) 


Figure  6-11.  Manually  Editing  Coordinates 
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Discrete  Scale 

Discrete-scale  indicators  provide  the  user  with  a  drop-down  list  to  enter 
the  value  rather  than  entering  an  actual  number.  For  example,  in  Figure  6-12,  the  user  has  the 
option  of  choosing  “Low”,  “Med”,  or  “High.”  The  assessed  value  given  to  this  indicator  is  the 
scale  value  associated  with  the  choice  the  user  makes  from  the  drop-down  list.  In  the  drop-down 
list  below,  if  the  user  selects  “Low”,  the  assessed  value  will  be  0.99.  If  the  user  selects  “Med”, 
the  assessed  value  will  be  0.66,  and  finally,  if  the  user  selects  “High”,  the  assessed  value  will 
be  0.33.  It  should  be  noted  that  in  this  example,  choosing  “Low”  is  a  good  thing.  The  edit 
screen  that  corresponds  to  this  example  is  shown  in  Figure  6-13  below. 


INPUT  INDICATOR  DATA  HERE 

Current  Status: 

■ 

1 

Choose  a  value  from 

Current  Value: 

High- 

-  Barriers  around  entrances 

Lf IB  U  1  UjJ 

-UUW1I  UUA 

3 


High-  Barriers  around  entrances 


Med--  PAP  cordon  around  entrances 
Low-  Normal  ops 


fubmit  Changes  &  View 


Figure  6-12.  Sample  Drop-Down  List  for  a  Discrete-Scale  Indicator 


Des  cr ipticm  §  c  ale  Value 

1. 

2. 

3. 

■Submit  Changes  |  View  Indicator  |  HOT  "Jit  View  Indicator  button  does-  not  EJVt  dunged 

New  Description  New  Scale  Value 

add  | 

Figure  6-13.  Edit  Screen  for  Example  in  Figure  6-12 


High--  Barriers  around  entrances 

0.33 

Med--  PAP  cordon  around  entrances 

0.66 

Low-  Normal  ops 

0.99 

Like  a  natural-scale  indicator,  the  assessed  value  is  derived  from  criteria 
set  forth  when  the  indicator  is  entered  into  the  system.  However,  the  assessed  value  isn’t  derived 
from  a  graph  or  curve,  it  is  entered  when  the  scale  items  are  added.  Figure  6-14  below  is  the 
editing  screen  that  appears  once  a  discrete-scale  indicator  is  created.  Notice  there  are  no  scale 
items  yet  but  there  is  an  area  at  the  bottom  to  add  new  scale  items.  Enter  the  information 
requested,  then  press  the  “add”  button  to  save  the  new  scale  item.  Figure  6-15  shows  the  results 
of  adding  a  scale  item  and  the  start  of  entering  another.  Figure  6-16  is  the  drop-down  list  the 
user  would  see  as  a  result  of  the  scale  items  added  in  Figure  6-15. 

Previously  entered  scale  items  can  be  changed  by  editing  the  description 
and/or  scale  value  then  pressing  the  “Submit  Changes”  button.  There  is  no  limit  to  the  number 


82 

UNCLASSIFIED 


UNCLASSIFIED 


EBASS  User’s  Guide 


of  scale  items  that  can  be  added.  Note :  the  current  version  of  EBASS  does  not  allow  the 
deletion  of  a  scale  item. 


Scale  Type: 


Discrete  Scale 


Change  Scale  Type  | 


No  scales  items  have  been  entered  yet. 

Add  scale  items  by  entering  the  information  requested 
then  press  the  "add"  button. 


Submit  Changes  View  Indicator 

NOTE :  Vimr  Indicator  button  doe-  not  save  changed  / 

New  Description 

New  Scale  Value 

Low-  No  reported  incidents 

0.1  |  add  | 

t 

t 

Enter  a  description: 

Enter  a  scale  value: 

The  description  is  the  text  that  is  presented  to 
the  user  in  the  drop-down  list  of  the  edit 
screen. 

The  scale  value  is  the  assessed  value  for  this 
indicator  if  this  scale  item  is  selected  from  the 
drop-down  list  by  the  user. 

The  text  should  be  descriptive  to  assist  the 
user  but  not  lengthy. 

The  scale  value  must  be  in  the  range  of  0  to  1 .0. 

Figure  6-14.  Adding  a  New  Scale  Item 


The  description  and 
scale  value  can  be 
changed  by  editing 
them,  then  pressing 
the  "Submit 
Changes"  button. 


i 


The  new  scale  item  has  been  added 


Description 


Scale  Value 


Low-  No  reported  incidents 

0.1 

|  NOTE  :  View  Indicator  button  doe  not  save  change  * 

New  Description 

New  Scale  Value 

Med  -  Less  than  5  incidents  reported 

.45 

add  | 

I 


An  additional  scale  item  ready  to 
be  added 


Figure  6-15.  A  New  Scale  Item  has  been  Added 
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Figure  6-16.  Drop-Down  List  Created  as  a  Result  Adding  Scale  Items 
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This  appendix  provides  a  series  of  diagrams  to  illustrate  the  basic  software  package  and 
database  structure  used  for  the  implementation  of  EBASS.  Figure  C.l  is  a  diagram  which 
provides  a  visual  depiction  of  the  EBASS  software  architecture.  This  is  the  general  structure  of 
the  software  components  of  the  tool  and  their  relationship  to  each  other. 


EBASS  Software 
Architecture  Diagram 


Web  Browser 


XHTML,  Javascript 
& 

Java  Server  Pages  (jsp) 


c 


Struts  Framework 
and  Java  Servlets 


Java  Classes 


Tomcat 

Web/App  Server 


Database 

(MySQL  or  Oracle) 


Figure  C.l.  EBASS  Software  Architecture  Diagram 


Figure  C.2  is  table  relationship  diagram  for  the  EBASS  database.  This  is  diagram 
provides  a  visual  depiction  of  the  tables,  the  elements  within  each  table,  and  the  reference 
relationships  between  each  table.  This  diagram  is  useful  in  allowing  user  to  understand  the  types 
of  data  stored  and  how  the  data  elements  relate  to  each  another. 
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Appendix  D:  Defining  Appropriate  Metrics 

“The  victorious  army  first  realizes  the  conditions  for  victory,  and  then  seeks  to  engage  in  battle.  The  vanquished 
army  fights  first  and  then  seeks  victory.” 

—Sun  Tsu,  The  Art  of  War 

D.l.  Executive  Summary 

Without  clear  objectives,  it  is  oftentimes  difficult  to  make  good  decisions.  To  come  up 
with  clear  objectives,  organizations  must  first  determine  what  is  important  and  then  figure  out  a 
way  to  assess  and  evaluate  how  well  they  are  able  to  perform  with  respect  to  what  is  important 
(United  States  Military  Academy,  2004).  As  the  familiar  saying  goes,  “What  gets  measured,  gets 
done.”  Metrics,  therefore,  play  a  critical  role  in  organizational  performance,  and  leaders  today 
are  very  concerned  about  exactly  what  is  measured  (Frost,  B.,  2000).  Regardless  of  the  type, 
size,  or  function  of  the  organization,  good  metrics  help  to  drive  systemic  improvement  and  are 
usually  (Brown,  M.,  1996): 

•  Few  in  number  to  allow  concentration  on  those  vital  key  variables 

•  Linked  to  key  business  drivers  to  promote  organizational  success 

•  Composed  of  a  mix  of  past,  present,  and  future  to  achieve  a  holistic  perspective 

•  Based  around  needs  of  customers,  shareholders,  and  key  stakeholders 

•  Driven  from  the  top  and  permeated  throughout  all  levels  of  the  organization 

•  Composed  of  multiple  indices  to  give  a  better  overall  assessment 

•  Flexible  and  adaptable  to  changes  in  environment  and  strategy 

•  Based  on  targets  or  goals  established  through  research  rather  than  arbitrary  numbers 

Most  organizations  spend  countless  hours  collecting  and  interpreting  data  intended  to 
enhance  business  performance  and  productivity.  Yet  a  large  portion  of  this  time  amounts  to 
nothing  more  than  wasted  effort  when  the  wrong  measurements  are  analyzed.  As  a  result, 
organizations  tend  to  create  unnecessary  procedures,  squander  resources,  and  detract  from  their 
objectives  when  this  happens.  Of  greater  consequence,  however,  is  that  wrong  metrics  oftentimes 
lead  organizations  to  inaccurate  decision-making  which,  in  turn,  translates  into  poor  choices. 


87 


This  section  of  the  paper  examines  how  organizations  can  ensure  that  they  are  focused  on  a  set  of 
appropriate  metrics  that  are  linked  to  critical  success  factors,  such  as: 

•  Reduced  risk  and  improved  safety 

•  Improved  operational  efficiency  and  effectiveness 

•  Increased  savings  in  personnel  training  costs 

•  Improved  stakeholder  satisfaction 

•  Better  relations  with  key  populations  within  an  area  of  interest 

•  Enhanced  command  and  control 

•  Improved  intelligence  capabilities  that  are  more  reliable,  timely,  and  accurate 

•  Increased  confidence  in  major  decisions 

Developing  appropriate  metrics  for  the  Effects-Based  Assessment  Support  System 
(EBASS)  is  critical  not  just  as  a  means  to  chart  progress  for  the  military  operation,  but  more 
importantly,  as  a  means  to  help  continuously  determine  how  specific  military  actions  are 
impacting  political  outcomes  and  decide  what  adjustments  are  needed  to  achieve  positive  end- 
states.  A  good  way  to  help  ensure  that  the  metrics  selected  for  EBASS  are  meaningful  is  to  make 
sure  the  chosen  measurements  are  SMART— specific,  measurable,  aligned,  realistic,  and  time- 
bound  (Moore,  C.,  2005). 

D.2.  Introduction — A  Shift  to  More  Meaningful  Metrics 

For  every  objective,  there  ought  to  be  at  least  one  quantifiable  metric  and  a  target  value 
for  that  metric.  Once  the  objective-level  target  for  a  metric  has  been  reached,  the  organization 
knows  that  a  particular  objective  has  been  achieved.  Unfortunately,  most  organizations  try  to 
measure  too  many  things,  measure  the  wrong  things,  use  measures  that  no  longer  have  relevance, 
or  use  measures  that  are  ambiguous  and  give  managers  and  workers  little  control  (Moore,  C., 
2005).  When  this  occurs,  a  great  deal  of  organizational  time,  energy,  and  resources  are  wasted. 

Michael  Dell,  CEO  of  Dell  Computers,  has  stated,  “To  motivate  an  employee  to  think 
like  an  owner,  you  have  to  give  her  metrics  she  can  embrace”  (Dell,  M.,  and  Fredman,  C.,  1999). 
SMART  metrics — those  that  are  specific,  measurable,  aligned  to  key  objectives,  realistic,  and 
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time-bound  (attainable  within  a  certain  time  period) — are  a  key  element  of  a  performance 
management  system  that  helps  link  employees  to  the  success  of  the  organization  (Figure  D.l). 
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*  Responsibility 

-  Work  environment 

*  Recognition 


Figure  D.l.  Aligning  Organizational  and  Individual  Goals  (Moore,  C.,  2005) 

However,  most  traditional  performance  management  systems  provide  “lagging”  rather  than 
“leading”  indicators.  They  tend  to  be  more  like  rearview  mirrors  that  explain  how  goals  were  or 
were  not  met  based  on  past  performance  rather  than  being  more  like  steering  wheels  that  enable 
organizations  to  adjust  to  changing  conditions  (Frost,  B.,  2000).  In  today’s  more  measure-based, 
goals-driven,  performance  management  culture,  improved  metrics  must  be  developed  to  better 
align  efforts,  implement  strategies,  and  focus  on  results  (Figure  D.2). 


THE  OLD  METRICS 

THE  NEW  METRICS 

Primarily  financial  . . 

. . .  Multidimensional 

After  the  fact  . . 

. . .  Mid-course  steering 

Control  and  report  . . 

. . .  Align  effort,  create  value 

Not  actionable  . . 

. . .  Line  of  sight  to  action 

Earnings  &  taxes  . . . 

. .  Implement  strategy 

Within  functional  areas  . . . 

. .  Manage  across  functions 

Manage  input  costs  .  * . 

. .  Manage  output  value 

Figure  D.2.  A  Shift  to  More  Meaningful  Metrics  (Frost,  B.,  2000) 

Organizations  that  make  a  concerted  effort  to  shift  to  metrics  that  are  multidimensional,  permit 
mid-course  steering,  create  value,  display  line  of  sight  to  action,  enhance  strategy,  facilitate 
management  across  multiple  functions,  and  assist  in  the  management  of  output  value  are  much 
more  likely  to  find  themselves  to  be  the  leaders  in  this  new  century.  The  shift  to  more 
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meaningful  metrics  will  lead  to  the  advancement  of  purposeful  organizations  that  succeed  at 
effectively  and  efficiently  getting  things  done  right.  EBASS  attempts  to  facilitate  such 
advancements. 

D.3.  A  Process  for  Linking  Meaningful  Metrics  to  Organizational  Objectives  for  EBASS 

Dr.  Bob  Frost  has  developed  a  three-step  methodology  for  helping  develop  meaningful 
metrics  and  performance  indicators  (Frost,  B.,  2000): 

•  Step  1:  The  organization  examines  its  business  strategy  and  stakeholders  to  find 
crucial  Performance  Topics. 

•  Step  2:  The  organization  determines  where  and  how  it  must  succeed  on  each  topic, 
spelling  out  the  where’s  and  how’s  as  a  set  of  Critical  Success  Factors. 

•  Step  3:  The  organization  considers  each  Critical  Success  Factor  and  defines  specific 
Performance  Indicators  that  will  track  success  on  it. 

This  three-step  process  works  from  the  general  to  the  specific — from  Performance  Topic  to 

Critical  Success  Factors  to  specific  Performance  Indicators  (Figure  D.3).  Taking  the  time  to  link 

performance  indicators  to  performance  topics  helps  to  ensure  the  chosen  metrics  will  be 

meaningful  and  provide  value  to  the  organization.  Raytheon  Corporation’s  Emery  Powell 

perhaps  summarizes  it  best  when  he  stated,  “A  strategy  without  metrics  is  just  a  wish.  And 

metrics  that  are  not  aligned  with  strategy  are  a  waste  of  time.” 


Figure  D.3.  Sample  of  Frost’s  3-Step  Process  for  Developing  Meaningful  Performance 

Indicators  (Frost,  B.,  2000) 


90 


EBASS  has  the  potential  of  distilling  large  quantities  of  data  and  information  into 
actionable  intelligence.  In  doing  so,  EBASS  empowers  commanders  with  the  power  to  choose 
appropriate  courses  of  action  that  advance  both  military  and  policy  objectives.  A  sample  three- 
step  metrics  development  chart  for  a  military  operation  is  shown  below  in  Table  D.  1 : 


Table  D.l.  A  Sample  3-Step  Metrics  Development  Chart  for  a  Military  Operation 


Performance 

Topics 

Critical 

Success 

Factors 

Suggested 

Performance  Metrics 
(leading) 

Old 

Performance  Metrics 
(lagging) 

Regional 

Stability 

Ability  of  governing  powers 
within  a  region  to  self- 
regulate 

Capabilities  of  coalition  forces 
within  a  region  +  capabilities  of 
local  forces  (an  examination  of 
unit  METLs);  amount  of  coalition 
forces  and  types  of  forces  needed 
in  area  of  operations 

Troop  strength  and  amount 
of  key  equipment  in  a 
particular  region 

The  functioning  of  key 
governing  responsibilities — 
health,  water,  food  supplies, 
power  generation,  etc. 

A  multi-dimensional  assessment  of 
key  governing  responsibilities 

Regional  request  for 
military  assistance  or 
support 

Neighboring  governments 
able  to  work  with  one 
another 

A  multi-dimensional  assessment  of 
key  regional  antagonisms  and 
mutual  support  structures 

Potential  and  actual 
regional  conflicts  and 
disputes 

Safety  of  Forces 

Providing  conditions  that 
enable  coalition  forces  to 
operate  under  better 
managed  risk  conditions 

A  multi-dimensional  assessment  of 
area  of  operations  based  on  key 
activities,  historical  events,  public 
sentiment,  locality,  etc. 

#  of  hostile  actions/period 

Capabilities  of  coalition  forces 
aligned  with  duties  and 
responsiblities  (METL  assessment) 

#  of  accidents/period 

#  of  fatalities/period 

D.5.  Conclusions:  EBASS  as  a  Viable  Strategy-Based  Performance  Management  System 

To  improve,  according  to  one  Webster’s  definition,  means  to  “enhance  in  value  or 
quality.”  This  definition  implies  that  value  and  quality  are  already  present  in  existing  methods  to 
measure  progress  and  success.  EBASS  aims  to  improve  upon  the  merits  of  current  military 
operational  assessment  methodologies.  Today’s  enabling  technologies  permit  EBASS  to  leverage 
the  large  amounts  of  information  within  an  area  of  interest  and  distill  it  into  a  comprehensible 
intelligence  that  is  timely,  reliable,  and  accurate.  Because  EBASS  takes  a  more  objective 
approach  to  characterizing  and  encapsulating  vast  amounts  of  data,  it  follows  that  EBASS  should 
be  an  ideal  strategy-based  performance  management  system  that  analytically  targets  continual 
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improvement  (Figure  D.4).  However,  success  for  EBASS  depends  largely  on  the  performance 
measures  chosen  to  gauge  the  success  of  military  actions  and  operations. 
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Figure  D.4.  Strategy-Based  Performance  Management  Design  (Frost,  B.,  2000) 

Perhaps  more  importantly,  continual  improvement  within  EBASS  itself  depends  largely  on  the 
development  and  selection  of  performance  indicators  that  are  SMART — specific,  measurable, 
aligned  to  key  objectives,  realistic,  and  time-bound. 

Jim  O’Brien,  University  of  Kentucky  Associate  Basketball  Coach,  once  remarked, 
“Excellence  is  the  unlimited  ability  to  improve  the  quality  of  what  you  have  to  offer.”  EBASS 
has  the  potential  to  enhance  the  way  the  military  assesses  operational  success.  With  the 
appropriate  metrics,  EBASS  has  the  potential  to  facilitate  real-time  communications  that  is 
required  of  highly  effective  and  efficient  organizations  (Figure  D.5).  With  the  right  metrics, 
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Organizations  that  Adaptively  Innovate  and  Consistently  Execute 


Bu  reau crat  ic  T rap 

Practices  become  overriding  concern  at  expense 
of  doing  what  is  best  for  the  organization 


Chaos  T rap 

No  tools  at  all  with  which  to 
leverage  and  generate  improvements 


Figure  D.5.  Avoiding  Both  the  Bureaucratic  and  Chaos  Trap 
(Brown,  S.,  and  Eisenhardt,  K.,  1998) 


Practices  become  overriding  concern  at  cost  of 
innovation,  creativity,  and  leadership 


The  lure  of  the  latest  and  greatest 
leaves  most  of  the  organization  behind 


Figure  D.6.  Avoiding  Both  the  Overconnect  and  Disconnect  Trap 
(Brown,  S.,  and  Eisenhardt,  K.,  1998) 

EBASS  has  the  opportunity  to  align  process  improvement  with  long-term  vision  in  order  to  make 
our  military  a  much  more  effective  instrument  for  policy  implementation  (Figure  D.6).  Ideal  for 
this  new  century,  EBASS  is  a  strategy-based  performance  management  system  that  aims  to 
achieve  operational  excellence. 
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